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ABSTRACT 


This  report  covers  continued  studies  toward  standardization  of 
coding  symbols  for  an  information  display.  Experiments  were 
directed  toward  locating  "the  most  readily  discriminated  five- 
symbol  code  complex"  as  measured  by  errors  and  location  times. 

Experiment  I  attempted  to  simplify  testing  procedures  by 
using  a  card  sorting  task.  The  same  five -symbol  code  was  presented 
as  a  black -on -white  simulated  display  in  Experiment  II.  Results 
were  not  comparable  and  the  simulated  display  was  used  in  further 
experiments  with  a  variety  of  codes. 
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with  the  population  of  shapes  within  the  code  complex.  Experimental 
results  led  to  general  recommendations  for  code  design.  Further 
studies  are  anticipated. 
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RADAR  SYMBOLOGY  STUDIES  LEADING  TO  STANDARDIZATION: 


II.  DISCRIMINATION  IN  MIXED  DISPLAYS 


INTRODUCTION 


The  use  of  radar  early  in  World  War  II  dramatically  increased  awareness  of 
on-going  activity  within  a  defended  area.  As  radars  became  more  sophisticated  and 
were  interfaced  with  computers,  the  radar  screen  became  a  prime  source  of  infor¬ 
mation  for  the  military. 

The  fundamentals  of  radar  search  are  simple.  A  narrow  beam  of  energy  is 
directed  into  space.  If  anything  is  present  in  its  path,  a  portion  of  this  energy  is 
reflected  back  to  a  receiver  at  the  source.  The  time  between  transmission  and 
return  of  the  reflected  energy  is  a  measure  of  the  distance  to  the  suspected  target 
while  the  horizontal  and  vertical  angles  of  the  transmitted  beam  reveal  information 
on  bearing  and  altitude.  Returns  are  impressed  on  a  variety  of  cathode  ray  tubes 
(CRT)  where  they  are  represented  as  disturbances  in  the  phosphor  coating  along  a 
time -based  sweep  line. 

The  most  informative  presentation  of  general  on-going  activity  is  the  planned 
position  indicator  (PPI).  This  display  is  a  polar  coordinate  representation  of  the 
area  with  a  time -based  sweep  line  revolving  about  the  antenna  position.  Any  object 
falling  within  the  transmitted  beam  is  seen  as  a  "pip"  at  the  appropriate  range  and 
bearing. 

Since  clouds  and  terrain  features  reflect  the  radar  beam,  these  returns  must 
be  separated  from  target  returns.  In  the  early  days  of  radar  all  analysis  of  this 
type  was  handled  by  the  radar  console  operator,  who  learned  to  recognize  "target 
signatures."  As  computers  were  wedded  to  the  radar  system,  much  of  the  integra¬ 
tion  and  analysis  of  data  could  be  machine -processed.  Later  it  became  possible  to 
add  "synthetic  information"  to  the  raw  returns  by  generating  a  geometric  form 
which  surrounded  a  particular  "pip."  This  form  identified  a  target  and  further 
»  simplified  the  operator's  task. 

Originally  only  a  few  geometric  shapes  could  be  generated  and  these  became  the 
usable  target  vocabulary.  As  new  methods  created  additional  forms,  the  symbol 
vocabulary  changed.  Unfortunately  this  change  was  not  represented  by  an  orderly 
growth  from  a  basic  word  list.  New  meanings  were  assigned  to  earlier  forms  and 
new  symbols  replaced  other  forms.  Bergum  and  Burrell  (8)  reported  difficulties 
created  under  time  stress  by  reversal  of  meaning  in  coding.  They  recommended 
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a  standardization  of  radar  symbols  before  the  problem  intensified.  Over  the  years 
the  need  for  standardization  has  been  continually  emphasized,  yet  new  codes  appear 
with  each  new  system.  Now  that  a  wide  variety  of  symbols  can  be  presented,  standard¬ 
ization  is  an  absolute  necessity. 

The  state  of  the  art  has  advanced  to  the  point  that  radar  screens  of  the  future 
can  be  expected  to  present  much  more  information  to  the  operator.  Synthetic  video 
produced  by  computer  analyses  of  radar  returns  can  transform  the  CRT  into  a 
meaningful  picture  of  action.  Targets  can  be  completely  identified  as  clear  symbols 
on  an  uncluttered  ground.  Map  overlays,  equipment  positions  and  terrain  features 
can  be  called  upon  demand.  The  radar  screen  will  thus  become  a  dynamic  informa¬ 
tion  display  presenting  the  entire  air  and  surface  action  over  a  large  area. 

There  is  a  temptation  to  build  large  vocabularies  of  meanings  based  on  informa¬ 
tion  which  the  computer  can  supply.  Gebhard  (31)  warns  that  such  complicated  codes 
are  of  no  value  unless  they  can  be  interpreted  by  the  operator.  Morgan  (48)  gives 
two  strict  rules  for  shape  coding:  (a)  shapes  that  are  compatible  with  and  have 
association  value  with  coded  objects  should  be  given  first  consideration;  (b)  only 
shapes  which  can  be  readily  discriminated  should  be  used  in  a  code  system. 

.  Baker  and  Grether  (4)  state  that  the  number  of  shapes  which  can  be  discriminated 
is  large  and  that  their  use  is  limited  only  by  space  and  ability  to  associate  symbol  and 
function.  Torre  and  Sander?  (56)  consider  association  values  within  the  air  defense 
picture  and  set  up  tentative  rules  for  symbol  design.  Davis  (22)  further  pursues 
stereotyped  meanings  and  suggests  that  the  circle  be  reserved  for  friendly  targets. 

She  advises  that  selection  of  other  symbols  within  a  code  complex  should  be  based 
on  how  readily  they  can  be  discriminated  and  how  readily  they  can  be  learned. 

Honigfeld  (41)  reviews  the  literature  on  coding  and  recommends  a  new  attack  on 
standardization.  Glaser,  Ramage  and  Lipson  (34)  warn  of  severe  penalties  in  accuracy 
when  a  display  contains  too  much  information.  As  a  rule-of-thumb,  they  advise  using 
only  as  many  symbols  as  are  absolutely  necessary.  They  suggest  that  Miller  (46) 
and  his  "magical  number  seven,  plus  or  minus  two,  "  should  be  considered  in  restricting 
the  parameters  of  a  code  design. 

The  Russian  engineering  psychology  literature  similarly  explores  methods  of 
encoding  information  but  offers  no  explicit  findings.  Leont’yeya  et  al  (45)  state  that 
the  capacity  of  a  machine  is  better  understood  than  the  capacity  of  its  operator.  They 
note  that  codes  are  designed  to  satisfy  the  needs  of  the  "planner"  with  little  attention  ' 
to  the  operator  who  will  decode  the  information.  Such  an  encoding  system  does  not 
endure  because  it  does  not  serve  its  purpose.  They  recommend  consideration  of 
speed  of  perception  as  a  measure  of  decoding  efficiency  and  add  that  code  design 
becomes  more  urgent  as  increased  amounts  of  information  become  available. 

With  the  need  for  standardizing  radar  symbology  so  clearly  demonstrated,  it  is 
difficult  to  understand  why  codes  continue  to  proliferate.  A  survey  of  the  literature 
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would  indicate  that  each  code  designer  leans  heavily  on  early  studies  which  seem 
significant  to  his  own  theory.  The  findings  of  these  studies  have  been  repeatedly 
summated  without  reference  to  the  original  articles  and  the  resulting  confusion  is 
of  little  value  in  ultimate  code  design. 

j 

Early  studies  on  visibility  of  geometric  forms  tend  to  creep  into  any  decision 
on  codes.  These  were  aimed  at  proving  or  disproving  the  Gestalt  theory  of  figure 
goodness  and  generally  argue  the  merits  of  the  circle  or  the  triangle.  There  is  a 
tendency  to  quote  these  studies  out  of  context  and  to  interweave  various  findings 
which  are  strictly  non -comparable.  Solid  and  linear  forms,  black  on  white  and 
white  on  black,  equal-area  and  equal -height  figures  are  lumped  together  without 
explanation  of  design.  A  review  of  the  most  quoted  studies  illustrates  the  problems 
inherent  in  a  literature  survey. 


Kleitman  and  Blier  (43)  found  a  triangle  superior  to  a  square,  circle  or  star. 
They  used  solid  black  equal -area  forms  against  a  white  ground  to  test  subjects  on 
peripheral  and  direct  vision.  Their  findings  in  the  two  testing  situations  are  closely 
related  and  appear  to  give  a  strong  preference  to  the  triangle.  Toward  the  end  of 
their  report  they  mention  that  all  four  targets  were  seen  at  about  the  same  angular 
subtense  (4  1/2  minutes)  in  a  walk-up  test.  The  equal-area  design  gives  the  triangle 
an  advantage  in  angular  subtense. 


Munn  and  Gcil  (49)  also  used  equal -area  forms  viewed  peripherally.  They 
report  that  the  triangle  was  recognized  over  the  greatest  perimetric  field  followed 
by  the  square,  circle,  rectangle  and  hexagon  in  that  order.  Angular  subtense  is 
again  a  possible  explanation  for  the  first  three  ranks.  It  is  also  obvious  that  only 
the  triangle  has  no  confusion  form  in  the  five -symbol  design. 


Collier  (17)  presented  seven  different  solid  forms  in  a  peripheral  view.  His 
sizing  method  is  not  described.  Scoring  was  in  terms  of  extent  of  perimetric  field 
and  a  subjective  measure  of  certainty.  He  reports  that  equilateral  and  isosceles 
triangles  are  vastly  superior,  followed  by  the  square,  parallelogram,  circle,  hexagon 
and  octagon  in  that  order.  Regardless  of  sizing  method,  it  is  apparent  that  confusion 
possibilities  are  greatest  between  the  circle,  hexagon  and  octagon.  Little  weight  can 
be  placed  on  the  reported  advantages  of  the  two  triangular  forms  which  were  identified 
merely  as  triangles. 

Helson  and  Fehrer  (39)  used  six  equal-area  solid  black-on -white  forms  includ¬ 
ing  a  circle,  half -circle,  square,  rectangle,  triangle  and  "square  with  a  small  square 
removed."  Their  measures  included  light  and  form  thresholds,  just  noticeable  form 
and  form  certainty.  The  triangle  and  rectangle  are  reported  as  the  best  forms.  The 
circle  did  not  place  first  on  any  measure  and  was  reported  as  "neither  good  nor  bad." 
All  advantages  could  be  explained  in  terms  of  angular  subtense  and/or  confusion  forms 
within  the  matrix. 
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In  a  similar  study,  Whitmor  (58)  presented  equal -area  solid  forms  as  peripheral 
displays.  He  also  reports  the  triangle  as  superior,  followed  by  the  diamond,  square, 
rectangle,  c'rcle  and  hexagon.  Again  angular  subtense  and  confusion  forms  are 
probably  biasing  results. 

Hochberg,  Gleitman  and  MacBride  (40)  approached  the  problem  from  a  new 
direction.  Their  targets  were  equal-area  circle,  square  and  St.  Andrew's  Crocs 
projected  as  bright  forms  of  ascending  intensities  on  a  dark  screen.  The  circle 
was  recognized  at  the  lowest  light  threshold  and  the  St.  Andrew's  Cross  required 
the  greatest  illumination  for  visibility.  No  triangle  was  included  in  their  study. 

In  this  case,  visibility  is  inversely  related  to  angular  subtense  in  direct  opposition 
to  other  equal -area  studies.  Conclusions  are  that  a  good  figure  is  simple,  compact, 
symetrical  and  familiar  as  expected  in  the  Gestalt  approach.  There  is  no  attempt 
to  justify  the  inverse  findings  in  studies  using  black  forms  on  a  white  ground.  The 
possibilities  of  a  decrease  in  "perceived  brightness"  over  larger  angular  extents 
are  not  considered. 

Hanes  (36)  suspected  apparent  brightness  as  a  factor  in  previous  equal -area 
experiments.  He  presented  circles,  equilateral  triangles  and  squares  in  three 
sizes  equal  in  area  to  a  1/16,  1/8  and  one-inch  diameter  circle.  While  his  illumina 
tion  boxes  and  controls  are  well  described,  the  targets  are  identified  only  as 
photographic  film  presentations.  The  discussion  suggests  that  he  used  bright  targets 
on  a  darker  surround.  Subjects  were  given  two  equal-area  targets  which  were  either 
alike  or  different.  One  figure  was  set  at  a  standard  brightness  (brightness  range 
0. 1  to  100  millilamberts)  and  the  subject  adjusted  the  other  to  match  brightness  of  the 
first.  Averaged  over  three  illumination  levels  the  triangle  had  the  greatest  apparent 
brightness  in  the  two  smaller  sizes.  This  situation  rever  ;d  for  the  largest  size 
(area  equal  to  one -inch  diameter  circle)  with  the  triangle  now  having  the  lowest 
apparent  brightness.  Hanes  suggests  that  at  this  size  the  subject  becomes  aware  of 
differences  in  maximal  linear  extent  and  interprets  the  greater  length  of  the  triangle 
as  a  brightness  decrement.  He  concludes  that  area,  perimeter,  visual  angle  and 
"apparent  size"  all  contribute  to  the  legibility  of  symbols. 

It  is  evident  that  these  much  quoted  studies  have  little  to  offer  in  the  search 
for  the  most  desirable  symbols.  Perhaps  the  most  valuable  findings  in  these  studies 
are  the  apparent  changes  in  visibility  of  an  individual  symbol  with  presentation  method 
and  the  suggestion  of  an  interaction  effect  when  highly  similar  characters  are  added 
to  a  limited  code  complex. 

Studies  using  a  larger  variety  of  symbols  also  fail  to  reveal  information  on 
the  characteristics  contributing  to  legibility.  Ferhaps  the  greatest  problem  in  these 
studies  is  caused  by  including  an  unbalanced  representation  of  form  families.  The 
circle  may  have  confusion  forms  represented  by  ellipses  and  a  variety  of  polygons 
while  linear  open  forms  may  have  no  possible  confusion  forms.  Unbalanced  designs 
of  this  type  are  i.’ghly  biased. 
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Casperson  (16)  is  frequently  quoted  by  the  anti  -Gestalt  group.  Reviews 
generally  state  that  lie  found  the  circle  and  ellipse  inferior  in  a  30-symbol  study. 

Examining  the  study  wc  find  that  Casperson  used  only  six  basic  symbols,  each 
proceeding  through  five  non-systematizcd  variations  which  tended  toward  a  progression 
from  a  solid  compact  form  toward  linearity.  Subjects  were  required  to  give  the 
family  name  fur  each  form  au  that  a  circle  was  reported  as  an  ellipse  and  a  square  i 

was  called  a  rectangle.  Raw  data  is  incompletely  recorded  and  results  are  averaged 
over  area  and  Height -to -width  ratios.  Conclusions  give  a  variety  of  rules  for  deter¬ 
mining  leg  ility  within  a  form  family.  Neither  the  report  nor  the  raw  data  is  of  i 

specific  u.-e  in  code  design.  j 

Sleight  (51)  included  21  geometric  solid  or  linear  black  forms  on  clear  plastic 
chips.  Symbol  sire  is  described  as  related  to  a  one -inch  diameter  circumscribed  i 

circle;  however,  his  illustrations  appear  to  be  based  on  height  alone.  Generally, 
solid  forms  were  used  with  a  heavy  emphasis  on  polygons.  A  cross  in  the  form  of  \ 

a  plus  sign  and  a  swastika  were  the  only  true  linear  characters.  Six  identical  I 

sets  of  each  form  were  included  for  a  total  of  126  symbol  chips  spread  as  a  mixed 
display  on  a  25-inch  white  circular  ground.  Subjects  were  scored  on  time  and 
accuracy  in  sorting  a  single  form  from  the  total  display.  Since  there  were  always 
exactly  six  targets,  search  time  was  to  some  effect  truncated  by  count.  Errors 
were  reported  as  negligible  and  were  not  considered  in  scoring.  Subjects  also  ranked 
the  forms  on  "attention-getting"  value.  Tne  swastika  ranked  first  on  both  time  and 
attention  rating.  The  circle  and  crescent  ranked  second  and  third  on  time  but  were 
rated  seventh  and  fifth  on  attention  value.  The  diamond,  triangle,  ellipse  and  square 
fell  near  the  middle  of  both  scoring  procedures.  The  trapezoid  and  four  polygons 
forms  were  poorest  on  botli  counts.  This  study  could  be  interpreted  as  giving 
definite  advantage  to  the  circle  ever,  in  the  presence  of  multiple  polygons  which 
approached  circular  form.  Since  the  symbols  were  a  full  inch  in  size  in  a  near 
point  task,  there  was  no  possibility  of  blur  in  a  subject  with  reasonable  visual 
acuity  and  confusion  elements  were  minimal.  This  study  does  seem  to  point  toward 
the  circle  as  a  good  symbol  in  a  mixed  display. 

Bowen,  Andreassi,  Truax  and  Orlansky  (13)  projected  linear  forms  which 
appeared  as  bright  strokes  on  a  darker  ground.  The  20  characters  in  this  series 
include  rounded  and  straight-line  closed  linear  forms  and  three  straight-line  open 
forms.  Symbol  size  is  described  as  based  on  a  5/8 -inch  circle.  Measurement  of 
the  illustrations  indicate  that  5/8  inch  was  the  major  dimension.  Since  viewing  was 
at  50  inches  this  size  represents  a  visual  angle  of  43  minutes  or  a  Snellen  acuity 
equivalent  of  about  20/15C.  Each  symbol  was  presented  twice  under  three  conditions 
of  noise,  two  conditions  of  blur  and  two  conditions  of  distortion  for  a  total  of  480 
presentations.  A  tachistoscopic  projector  exposed  each  symbol  for  one -half  second. 

The  subject  identified  the  symbol  on  a  paper  matrix  showing  all  possible  views. 

Scoring  was  based  on  a  confusion  matrix.  Results  are  reported  averaged  over  all 
conditions,  first  as  the  probability  of  correct  identification  of  an  individual  symbol, 
and  then  as  an  "articulation"  score  for  sets  of  symbols  of  various  sizes.  This  series 
has  been  widely  used  in  code  design  on  the  basis  that  the  symbol  sets  recommended  in 
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this  report  had  been  used  in  a  simulated  display.  Actually  only  one  set  of  four 
symbols  was  presented  as  a  simulated  display  and  that  set  did  not  conform  to 
recommendations.  In  fact  one  of  the  symbols,  a  plus  sign  with  two  short  vertical 
modifiers,  had  proved  so  unreliable  in  the  first  experiment  that  its  use  was 
negatively  recommended.  It  was  this  symbol  which  rated  highest  on  the  simulated 
display  experiment. 

The  simulated  display  consisted  of  a  group  of  photographic  negatives  mounted 
on  a  circular  trans illuminated  screen.  In  each  presentation  of  40  characters,  an 
individual  symbol  appeared  eight  to  13  times.  The  subject  counted  the  number  of 
designated  targets.  Scoring  was  on  the  basis  of  location  time  plus  a  derived 
correction  time  related  to  errors.  The  modified  plus  sign  gave  the  lowest  time  score 
followed  by  circle,  square  and  triangle  in  that  order.  The  authors  suggest  that  the 
advantage  to  the  plus  sign  may  have  been  related  to  the  fact  that  it  was  the  only  open 
figure.  Another  complicating  factor  is  noted.  The  symbols  were  not  held  to  a 
standard  orientation  but  were  symmetrical  about  the  radius  of  the  display  circle. 

The  most  valuable  finding  is  the  final  superiority  of  the  modified  plus  sign,  which 
was  poorly  discriminated  as  an  isolated  symbol  in  the  previous  experiment  but 
stood  out  clearly  as  a  different  form  amid  a  group  of  enclosed  figures.  This  finding  is 
especially  valuable  because  it  demonstrates  the  importance  of  the  context  in  which 
a  symbol  appears  in  determining  how  well  it  can  be  discriminated. 

Williams  and  Falzon  (59)  presented  100  different  forms  in  mixed  linear  and 
solid,  symmetrical  and  non-symmetrical,  nonsense  and  pictorial  form  families. 
Individual  symbols  were  presented  tachistoscopically  for  one -half  second  as  a 
bright -on-dark  display.  Subjects  located  the  viewed  symbol  on  a  black -on -white 
pape.  matrix  series.  Scoring  was  based  on  accuracy,  confusion  errors,  search 
time  and  numerous  other  variables.  The  investigators  found  that  they  had  included 
too  many  factors  for  efficient  analysis  but  were  able  to  isolate  simple  geometric 
or  pictorial  forms  as  advantageous. 

In  a  second  study  Williams  and  Falzon  (60)  reduced  their  matrix  to  a  circle, 
triangle,  square  and  90°  diamond  with  horizontal,  vertical  or  diagonal  linear 
modifiers  extending  beyond  the  symbol  or  contained  within  its  borders.  Using 
scoring  and  projection  techniques  of  the  earlier  study,  they  found  the  circle  and 
square  superior  to  the  diamond  and  triangle  and  also  determined  that  modifiers 
contained  within  the  symbol  led  to  fewer  confusion  errors  than  extended  modifiers. 

The  latter  finding  appears  useful. 

These  studies  using  a  large  variety  of  symbols  influenced  selection  of  codes 
in  later  experiments.  All  have  been  interpreted  out  of  context  and  beyond  the  scope 
of  their  experimental  design.  None  presented  a  realistic  display  situation  with  the 
possible  exception  of  the  second  experiment  of  Bowen  et  al  (13).  In  this  case  the 
conditions  of  the  second  experiment  including  only  four  symbols  which  did  not  conform 
to  their  recommendations  have  tended  to  be  transferred  to  the  20-symbol  experiments. 
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True  radar  simulator  studies  are  limited  in  number  and  scope.  They  tend  to 
point  out  equipment  problems  rather  than  to  give  information  on  symbol  form. 

Baldwin,  Wright  and  Lehr  (6)  presented  raw  radar  pips  moving  across  a 
screen  at  speeds  equivalent  to  500  to  900  knots  using  a  clear  screen  and  also 
simulated  Electronic  Counter  Measures  (ECM)  noise.  Objects  were  trained  under 
four  conditions:  (1)  the  target  is  brighter  than  the  noise,  (2)  the  target  has  a 
distinctive  shape,  (3)  the  target  has  greater  brightness  and  also  a  distinctive  shape 
and  (4)  all  information  is  obtained  through  passive  watching  of  a  presentation. 
Subjects  trained  to  watch  for  shape  only  did  best  on  a  clear  screen.  With  ECM  the 
combined  shape  and  brightness  group  was  superior. 

Baitlet  and  Williams  (7)  ran  a  similar  study  and  found  that  brightness  was  the 
best  clue  on  a  clear  screen  while  pattern  discrimination  was  the  major  factor  with 
ECM.  This  study  may  have  initiated  interest  in  pattern  recognition  through  noise. 

Blair  (11)  evaluated  three  code  complexes  (four  symbols  in  each  code)  using 
the  few  forms  available  at  the  time.  He  found  that  codes  including  two  rotations  of 
half  or  three-quarter  circles  caused  difficulty.  His  best  code  had  the  least  similar¬ 
ity  between  symbols. 

Crook  et  al  (18,  19)  presented  a  limited  character  display  at  a  variety  of 
magnifications.  They  found  forms  subtending  13  to  25  m  mutes  of  arc  superior  to 
larger  or  smaller  characters.  Scoring  was  based  on  time  and  accuracy. 

More  closely  related  to  a  symbolic  display  are  the  studies  of  Gerathewohl  and 
Rubenstein  (33).  Using  an  Air  Force  Simulator  they  presented  solid  equal-area 
(70mm)  triangles,  circles,  squares  and  St.  Andrew’s  crosses  on  a  PPI  scope.  The 
targets  appeared  as  solid  bright  figures.  There  was  some  evidence  that  the  circle 
was  a  superior  target;  however,  the  equipment  broke  down  before  the  experiments 
were  concluded. 

Gerathewohl  (32)  repeated  these  studies,  adding  a  variable  contrast.  The 
studies  included  both  clear  screen  and  ECM  simulation.  In  these  studies  the  triangle 
was  the  most  frequently  identified  target;  however  the  triangle  was  named  more 
frequently  and  hence  had  a  high  confusion  score.  While  conclusions  were  guarded 
in  both  studies,  Gerathewohl  has  been  quoted  by  proponents  of  both  the  circle  and  the 
»  triangle. 

Horton  (42)  also  used  solid  bright  targets  on  a  PPI  scope  for  a  task  of  reading 
range  and  bearing  of  a  specific  form.  His  characters  were  the  circle,  eclipse  and 
a  solid  arc  closely  resembling  a  trapezoid.  The  forms  were  "equated  for  perceived 
area. "  Sizes  or  area  are  not  given.  Shape  had  no  effect  on  speed  or  accuracy  of 
reading  coordinates. 
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Dardano  and  Donley  (20)  projected  bright  line  symbols  on  a  phosphor -coated 
screen.  Their  targets  were  a  circle,  cross,  cross  within  circle,  open  half  circle 
and  open  three-quarter  circle.  Size  and  stroke  width  are  not  given.  They  presented 
either  24  or  48  symbols  with  the  proportions  of  individual  forms  systematically 
varied.  Subjects  marked  each  symbol  group  and  were  scored  on  time  and  accuracy. 
The  cross  within  circle  rated  best  followed  by  cross,  circle,  half  circle  and  three- 
quarter  circle.  The  three-quarter  circle  accounted  for  most  of  the  confusion  errors 
and  was  considered  a  poor  target.  Taking  their  findings  together  with  the  stereo¬ 
typed  meaning  studies  of  Torre  and  Sanders  (56)  they  suggested  the  cross  within  a 
circle  as  a  hostile  target,  the  circle  as  a  friendly  interceptor  and  the  cross  as 
unknown. 

Dardano  and  Stephens  (21)  did  a  follow-up  study  using  the  same  series  of 
symbols  with  the  three-quarter  circle  omitted.  They  also  replaced  the  projector 
with  a  symbol  generator.  Size  of  the  basic  circle  or  cross  was  varied  in  1/16 -inch 
increments  within  the  1/8  to  1/2 -inch  size  range.  Stroke  width  is  not  given.  They 
followed  the  scoring  procedures  of  the  earlier  study,  and  their  results  now  indicated 
that  the  cross  within  the  circle  was  the  poorest  symbol  and  the  circle  was  superior. 
They  noted  a  brightness  advantage  for  the  circle  and  half  circle  created  by  the 
generation  method  which  might  have  influenced  results;  however,  they  felt  the 
omission  of  the  three-quarter  circle  created  new  problems.  They  conclude  that 
strong  similarities  of  symbols  are  undesirable. 

Fried  (28,  29)  continued  studies  on  these  four  symbols  under  a  variety  of 
continuous -wave  noise  conditions,  simulating  ECM  effects.  He  now  found  that  the 
cross  significantly  superior  to  the  other  three  symbols  in  clear  or  noisy  displays. 
There  were  no  significant  differences  between  the  circle,  half  circle  and  cross 
within  the  circle.  He  had  used  a  new  symbol  generator.  A  review  of  his  equipment 
indicated  that  the  cross  had  a  strong  brightness  advantage  in  this  system. 

This  series  of  studies  tends  to  put  some  emphasis  on  apparent  brightness  as 
a  legibility  advantage.  It  also  points  out  that  changes  in  display  methods  or  in  the 
symbol  series  may  create  invalid  or  non-reproducible  results. 

Harris,  Green,  Wilson  and  Liaudansky  (38)  studied  the  design  of  charactron 
symbols  for  CRT  displays,  following  an  orderly,  step-wise  experimental  design. 
Preliminary  testing  used  a  flying-spot  scanner  to  simulate  characters.  Twenty -one 
special  geometric  symbols  of  linear  and  solid  forms  were  projected  as  bright  on  a 
dark  ground.  Symbols  included  both  geometric  forms  and  pictorials  which  were 
assigned  simple  meaningful  names.  Subjects  were  over -trained  in  symbol  naming 
using  a  flash -card  presentation.  In  the  final  testing  the  symbols  were  presented 
singly  at  five -second  intervals  near  the  center  of  a  PPI.  Subjects  named  each  symbol 
as  presented.  There  were  no  corrections  of  errors.  Scoring  was  based  on  a 
confusion  matrix.  The  authors'  conclusions  indicate  that  variations  of  a  single  form 
family  should  be  avoided.  Symbols  within  a  coding  alphabet  should  be  as  unlike  as 
possible. 
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Pattern  recognition  studies  have  attempted  to  identify  the  characteristics 
which  make  forms  perceptually  different.  To  date  their  definite  contributions 
have  been  minor  and,  generally,  poorly  understood. 

Baker,  Morris  and  Steedman  (5)  created  nonsense  forms  by  random  filling 
of  a  square  matrix.  The  forms  varied  greatly  in  area,  visual  subtense  and  the 
number  of  changes  in  direction  of  the  peripheral  outline.  They  conclude  that  the 
best  measure  of  a  good  pattern  is  the  ratio  of  the  target  (T)  area  to  the  area  (A) 
of  circumscribing  circle.  If  T/A  is  small  the  target  is  located  more  rapidly  and 
with  fewer  errors.  By  this  formula  a  circle  would  be  the  poorest  possible  target. 

Steedman  and  Baker  (54)  pursued  this  study  further  adding  blur  and  size 
differences.  They  found  blur  within  the  range  of  their  tests  had  no  effect  if  the 
displayed  reference  standard  was  equally  blurred.  Visual  subtense  was  strongly 
related  to  legibility  in  sizes  of  20  minutes  or  less. 

Boynton  (14,  15)  presented  a  single  critical  form  of  angular  construction 
against  a  background  of  irregular  curved  nonsense  forms  of  similar  size.  On 
some  exposures  no  critical  form  was  presented.  Subjects  were  on  an  irregularly 
presented  monetary’  reward  schedule  to  reduce  guessing  in  a  boring  situation. 

This  study  and  numerous  subsequent  experiments  in  the  same  laboratory  are 
directed  toward  identifying  factors  that  influence  the  visibility  of  a  form. 

Generally  findings  have  been  related  to  contrast,  density,  observation  distance 
and  exposure  time,  factors  that  have  been  fairly  well  defined  previously. 

Dccsc  (24)  approached  pattern  recognition  with  random  straight-line  forms 
generated  withm  a  two -inch  diameter  circle.  He  classed  his  targets  as  simple  or 
complex,  regular  or  irregular.  Complexity  was  based  on  the  number  of  changes 
in  direction.  Regular  forms  had  only  right -angle  changes  while  irregular  forms 
contained  varying  angles.  The  subject  was  required  to  locate  a  demonstrated 
target  from  a  set  of  five  similar  nonsense  forms.  Results  are  not  clear  cut.  The 
more  complex  forms  with  right  angle  turns  are  best  if  only  one  form  needs  to  be 
remembered.  Performance  in  general  suggests  that  subjects  do  best  when  the 
nonsense  form  reminds  them  of  something. 

French  (26)  presented  dot  patterns  on  a  noise -degraded  CRT.  ! ...  report 
concludes  that  the  more  dots  included  in  the  pattern,  the  greater  the  p-j.  ‘■■liny  of 
detection.  Gaito  (30)  finds  straight  lines  superior  to  curved  forms  and  fee's  angles 
give  much  information. 

Mooney  (47)  performed  a  recognition-recall  experiment  with  irregular  black 
and  white  forms  and  concluded  that  pattern  learning  comes  from  a  single  glance 
rather  than  prolonged  inspection,  Fitts,  Weinstein,  Rappaport,  Anderson  and 
Leonard  (25)  generated  nonsense  figures  on  an  8  x  6  matrix,  using  a  probability 
formula  for  generation.  In  a  recognition  test  the  totally  symmetrical  figures  and 
those  symmetrical  about  a  vertical  axis  were  superior  to  asymmetrical  figures 
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and  those  symmetrical  about  a  horizontal  axis.  Booth  and  Glorioso  (12)  report  a 
CRT  computer -generated  patterning  device  that  will  permit  further  study  of  these 
factors.  Murray  (50)  suggests  that  symmetry  may  have  a  time  advantage  in  reducing 
transformation  requirements  within  the  visual  pathways. 

Attneave  (2)  and  Attneave  and  Arnoult  (3)  used  irregular  solid  forms  plotted 
from  a  random -numbers  table  in  an  attempt  to  identify  the  factors  contributing  to 
judged  complexity  of  form.  They  find  that  asymmetry,  angular  variability,  the 
number  of  sides  and  the  ratio  of  the  square  of  the  perimeter  to  the  total  area  can 
be  weighted  to  produce  a  formula  which  accounts  for  90  percent  of  judged  complexity. 
This  research  is  also  continuing. 

The  status  of  knowledge  in  code  selection  can  be  summed  up  by  the  1957  National 
Research  Council  report  of  its  meeting  on  form  discrimination  as  related  to  military 
problems.  White  (57)  of  the  U.  S.  Navy  described  the  electronic  displays  of  the 
various  services  and  the  conglomeration  of  codes  that  had  sprung  up  with  each  new 
display.  He  asked  for  a  documented  standardization  of  symbolic  codes  before  the 
situation  was  completely  out  of  hand.  Harker  (37)  of  the  U.  S.  .Army  asked  for 
realistic  display  studies  related  to  actual  jobs,  using  the  full  dimensions  of  the 
natural  environment.  He  stated  that  measurements  of  psychological  functions  which 
must  be  tenuously  related  to  the  task  have  little  utility.  Bersh  (9)  of  the  U.  S.  Air 
Force  noted  that  studies  on  forms  have  been  too  heavily  loaded  on  single -form 
visibility.  For  rapid,  accurate  decoding  of  information,  he  felt  the  influence  of 
symbols  on  other  symbols  within  a  code  complex  should  be  carefully  analyzed.  These 
representatives  of  the  armed  services  all  emphasized  the  importance  of  time  and 
errors  in  decoding  displays. 

Blackwell  ( 10)  immediately  stated  that  current  knowledge  was  probably  of  little 
use  in  solving  any  practical  problems  of  the  military.  After  the  papers  had  been 
completed,  he  repeated  this  statement  and  added  that  the  studies  reported  bore  no 
resemblance  to  actual  situations  in  life. 

Smith  (53)  reviewed  the  historical  stages  of  form -discrimination  studies  for 
the  meeting.  The  Italian  artists  of  the  Rennaisancc  in  their  attempts  to  create  two- 
dimensional  representations  of  a  three-dimensional  world  gave  psychology'  the  original 
information  on  monocular  cues  to  distance  and  original  notions  of  form  discrimination. 
The  assoc iationists  put  their  emphasis  on  sensations  and  local  signs.  The  Gestalt 
school  was  first  concerned  with  total  impression  of  the  observer.  Smith  referred  to 
the  Gestalt  laws  as  "mainly  Monday  morning  quarterbacking,  "  and  added  that  the 
major  contribution  of  this  period  is  the  "initial  formulation  of  problems  which  are  as 
yet  unsolved. "  He  described  the  present  era  as  an  attempt  to  cope  with  urgent 
practical  problems  necessary  for  survival  of  the  species.  This  era,  he  said,  is  in 
its  infancy  and  its  direction  is  still  poorly  defined.  Investigation  of  nonsense  forms 
is  aimed  at  uncovering  the  basic  factors  which  make  a  form  visible  or  at  designating 
an  equation  of  visibility.  While  these  studies  may  be  of  great  value  in  terms  of 
learning,  they  do  not  meet  the  immediate  practical  problems  of  the  military.  Smith 
suggested  attention  to  forms  which  might  actually  be  used  for  identification. 
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Tanner  (55)  discussed  form  perception  as  related  to  information  theory.  He 
felt  we  must  know  much  more  about  the  environment,  the  operator,  and  the  task 

before  we  can  designate  the  "channel  capacity"  in  any  complex  viewing  situation.  j 

Arnoult  (1)  told  the  group  about  his  pairing  method  for  studying  angular 
nonsense  forms  and  rating  symbols  on  complexity.  Boynton  (14)  criticized  this 
method  for  failure  to  include  a  full  population  of  confusion  forms.  He  then  described 
his  own  procedure  which  presented  only  one  angular  target  within  a  large  population 
of  curved  irregular  forms.  Neither  study  provided  information  for  choice  of  a  code. 

Debons  (23)  noted  difficulties  in  translating  academic  information  into  data 
useful  in  other  fields.  He  suggested  an  extensive  literature  review.  Sleight  (52), 
who  had  suggested  simulation  as  an  ideal  method  for  studying  man -machine  perfor¬ 
mance,  doubted  the  usefulness  of  a  literature  search.  He  argued  that  psychologists  ‘ 

do  not  write  or  speak  in  a  manner  that  communicates  their  meaning  to  colleagues. 

Under  such  circumstances  he  felt  the  literature  had  little  meaning  for  others.  The 

discussion  periods  lent  weight  to  this  argument.  No  one  appeared  to  understand 

the  theory  behind  any  study  other  than  his  own.  ; 

i 

Arnoult  (1)  suggested  that  the  group  members  had  two  general  theories  of 
attack.  He  felt  that  people  interested  in  interpretation  of  radar  screens  and  other 
practical  displays  need  to  gear  their  studies  to  periods  of  five  to  10  years  because 
of  engineering  obsolescence.  Psychologists  interested  in  the  basic  principle  of  form 
perception  are  working  within  a  time  frame  of  800  years  or  perhaps  until  the  human 
being  is  obsolete.  He  questioned  whether  the  Armed  Forces  were  interested  in 
supporting  such  long-term  research.  This  question  seems  to  have  been  answered: 
the  nonsense  form  studies  have  continued  and  multiplied,  and  the  military  still  does 
not  have  the  answer  to  its  problems. 
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SUMMARY 


Any  new  research  problem  traditionally  starts  with  a  literature  review. 
Theoretically,  this  gives  good  background  information  and  points  out  a  general 
avenue  of  approach  for  further  studies.  In  many  instances  the  building  blocks  for 
a  research  design  are  located  in  reports  of  earlier  studies.  At  other  times  a 
stereotyped  pattern  may  develop  a  "stumbling  block  to  the  truth." 

Our  literature  search  did  not  uncover  information  closely  related  to  symbolic 
coding  of  information  displays.  Perhaps  it  would  be  more  appropriate  to  say  that 
no  studies  were  oriented  toward  decoding  the  display.  Emphasis  was  on  locating 
"good  figures"  from  the  Gestalt  definition  and  proving  or  disproving  their  effective¬ 
ness. 


There  was  a  large  amount  of  controversial  data  on  the  relative  discernibility, 
or  pure  visibility  of  ti.e  circle,  square  and  triangle.  Codes  were  then  logically 
derived  from  discernibility  findings.  The  total  codes  were  generally  not  tested. 

A  few  studies  did  consider  the  possibility  that  two  or  more  highly  visible 
forms  might  be  too  similar  for  use  in  a  mixed  display.  None  of  these  studies  of 
inter -symbol  discriminability  used  a  display  with  a  large  variety  of  symbols 
scattered  randomly  over  an  area.  In  several  instances  three  or  four  symbols 
were  studied  in  a  very  realistic  design  on  a  simulated  CRT.  but  in  all  instances 
equipment  failed  or  character  generation  favored  one  symbol  and  conclusions 
were  guarded. 

Any  realistic  display  of  mixed  symbols  mu  it  be  carefully  designed  to  avoid 
positional  advantages  and  patterning  cues.  A  single  code  complex  requires  a 
number  of  display  drawings  and  each  code  change  represents  a  complete  set  of 
these  drawings.  The  drafting -board  work  is  tedious  and  exacting.  Perhaps  this 
work  accounts  for  the  fact  that  earlier  inter -symbol  confusion  studies  employed 
simpler  designs.  While  these  techniques  were  flexible,  they  do  not  seem  to 
simulate  or  represent  an  information  display.  Usually  subjects  became  familiar 
with  a  total  test  battery  and  were  then  given  the  task  of  sorting,  selecting  or  naming 
symbols  individually  presented  on  cards  or  by  tachistoscope .  Performance  scoring 
usually  included  some  timing  element.  Inter-symbol  discrimination  scores,  however, 
were  almost  totally  dependent  on  errors. 

Tachistoscopic  tests  were  especially  unsatisfactory,  since  they  required 
repetitive  test  runs  with  the  subject  naming  or  defining  the  target.  Data  from  these 
studies  suggests  that  subjects  developed  a  preference  for  a  particular  symbol  and 
named  it  more  frequently.  Using  any  method  of  limits,  this  preferred  target 
reached  the  criterion  of  correct  identification  at  either  a  smaller  size  or  faster 
exposure.  This  same  target  also  appeared  as  the  most  frequent  confusion  figure 
for  other  symbols.  Regardless  of  the  mathematical  manipulations  used  to  correct 
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for  this  factor,  results  were  definitely  tempered  by  the  preference  element. 
Tachistoscopic  studies  also  generally  used  a  seven  to  ten-foot  working  distance 
since  projectors  are  designed  for  a  fairly  long  slide -to -screen  separation.  Under 
these  conditions  there  was  little  resemblance  to  the  radar  operator's  task. 

Sorting  procedures  were  near -point  tasks  and  had  the  advantage  of  requiring 
some  perceptual -motor  coordination.  While  card  manipulation  was  in  no  way 
related  to  the  manual  requirements  at  a  PPI  console,  the  sorting  procedure  did 
interweave  with  the  viewing  process  and  provided  a  better  task  simulation. 
Unfortunately,  sorting  cards  into  individual  categories  does  not  permit  timing  on 
individual  symbols.  Studies  using  these  techniques  reported  only  inter-symbol 
confusion  errors  and  total  sorting  time.  In  most  instances  the  time  element  was 
considered  as  a  measure  of  increased  task  complexity  with  additional  symbols. 

Studies  designed  on  separation  of  an  individual  symbol  from  a  complex  card 
deck  reported  errors  of  confusion,  errors  of  omission  and  time  for  each  category. 
This  technique  had  all  the  advantages  of  the  sorting  methods  and  provided  more 
useful  data.  It  is  the  most  flexible  of  the  simple  design  methods.  A  single  symbol 
may  be  replaced,  densities  may  be  varied  and  additional  categories  may  be  added 
with  minimal  effort.  For  our  studies  this  method  was  chosen  for  comparison  with 
a  more  realistic  display.  If  findings  provided  comparable  data,  the  simple 
technique  would  be  the  method  of  choice  on  further  experiments. 
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EXPERIMENT  I  -  SYMBOLS  IN  ISOLATION 


Studies  of  discriminabiiity  involving  a  number  of  symbols  have  leaned  heavily 
on  sorting  or  similar  techniques.  The  subject  is  given  a  variety  of  symbols  on 
individual  exposures  and  is  asked  either  to  sort  them  into  categories  or  to  separate 
a  single  type  from  the  total.  The  procedure  is  simple  and  requires  a  minimal 
design  effort,  but  it  may  be  questioned  whether  the  confusion  effect  of  mixed  symbol 
types  in  a  single  display  is  reproduced  by  this  method. 

Since  sortirg  techniques  would  permit  the  greatest  flexibility  in  testing,  initial 
efforts  employed  this  procedure.  Tests  were  designed  to  present  symbols  on  individ¬ 
ual  cards  within  a  mixed  deck.  There  was  no  distracting  material  on  a  given  card. 
Confusion  elements  were  present  only  as  the  total  variety  of  similar  forms  within 
the  test  decks.  Final  testing  required  individual  sorts  on  two  dimensions  --a  basic 
symbol  and  a  modifying  element.  The  procedure  is  similar  to  sorting  playing  cards 
by  suit  and  face  value. 

In  all  instances  two  unbalanced  card  decks  were  used  to  permit  varying  item 
counts  with  a  constant  total.  With  this  procedure,  card -handling  time  was  equalized 
and  search  was  directed  toward  an  individual  item  without  the  subject  knowing  the 
total  count. 


Familiarization  Procedure 

Preliminary  familiarization  material  consisted  of  non-standard  playing  card 
decks  with  varying  suit  and  face -value  counts.  In  the  first  task  the  subjects  simply 
counted  the  card  decks  as  rapidly  as  possible.  Since  all  subjects  were  familiar  with 
playing  cards  the  55  count  invariably  brought  questions.  Subjects  were  told  that  they 
were  using  special  sets  of  cards  made  up  from  several  decks.  This  prepared  them 
for  their  second  task,  a  timed  sorting  of  the  Kings.  There  were  six  Kings.  Sorting 
by  face  value  or  suit  continued  with  alternating  decks  until  sorting  times  reached  a 
plateau.  At  this  stage  all  subjects  handled  the  cards  well  and  had  become  aware  of 
the  fact  that  counts  varied  from  suit  to  suit  and  deck  to  deck. 

Additional  familiarization  included  counting  two  unbalanced  decks  of  blank  3x5 
cards  with  the  two-deck  total  equalling  110.  This  practice  gave  experience  in  handling 
the  material  used  in  actual  testing.  Counts  were  timed  until  this  two-deck  count  also 
reached  a  plateau. 

To  give  the  experimenters  practice  in  perfecting  test  techniques,  all  procedures 
were  initially  run  on  casual  troops  who  would  not  be  available  for  the  entire  test 
session.  Thus  both  subjects  and  experimenters  entered  the  test  sessions  with 
appropriate  basic  skills. 
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Subjects 


Subjects  were  rougly  matched  on  the  basis  of  playing-card  sort  and  blank -deck 
counting  times  and  errors.  Improvement  in  performance  during  the  familiarization 
period  was  considered  as  a  learning  trend  and  was  included  in  the  subject -matching 
procedure.  Four  groups  of  seven  subjects  were  identified  in  this  initial  procedure. 

All  potential  subjects  were  tested  on  the  Bausch  and  Lomb  Ortho -Rater  for 
visual  acuity,  far  and  near.  Two  had  scores  of  8  (20/25)  and  the  remainder  scored 
10  (20/20)  or  better  on  both  far  and  near.  The  age  range  was  19  to  52.  Civilian 
and  military  technical  employees  within  the  laboratory  provided  our  subject  pool. 


Pre-Testing 

Since  a  doable -fatigue  testing  order  was  desirable,  four  groups  of  matched 
subjects  were  needed.  The  pre-tests  were  designed  to  group  subjects  by  speed 
and  accuracy  for  the  final  tests.  Alpha-numeric  symbols  were  used  in  this  phase 
to  avoid  a  practice  effect.  Otherwise,  the  pre-test  material  followed  the  final  test 
design  exactly. 

Test  material  was  centered  within  t"  1/4 -inch  radius  on  3  x  5  cards.  Each 
sub-test  consisted  of  two  55 -card  decks.  The  five  basic  symbols  or  suits  of  a  series 
appeared  from  9  to  13  times  in  each  deck  for  a  two-deck  count  of  22.  Modifying 
symbols,  or  face  values,  appeared  3  to  8  times  in  a  deck  for  a  two -deck  count  of  11. 
While  the  test  decks  were  used  and  timed  alternately,  item  sorts  were  inversely 
ordered  to  avoid  possible  recognition  of  the  total  two -deck  count.  Times  and  errors 
were  later  combined  for  individual  items  over  the  two  decks  and  are  strictly  compar¬ 
able  . 


In  the  pre-test  series,  10 -point  capital  letters  A  through  E  represented  suits 
and  10-point  numerals  1  through  9  and  a  blank  space  represented  face  values.  Three 
sets  of  test  decks  were  included:  alphabetic  alone,  numeric  alone,  and  an  alpha¬ 
numeric  combination.  In  the  combined  series,  the  numeral  or  blank  always  followed 
the  letter.  Symbols  were  typed  on  the  cards  within  a  t  1/4 -inch  radius.  The  upper 
left-hand  corner  was  cut  from  each  deck  to  assure  proper  letter  orientation  after 
shuffling.  A  segmented  linear  code  on  the  reverse  side  of  the  test  cards  aided  the 
tester  in  rapid  identification  of  errors. 

There  was  no  emphasis  on  either  speed  or  precision.  From  the  earliest 
familiarization  series  the  experimenter  used  the  stopwatch  in  a  position  where  it 
could  be  easily  read  by  the  subject  in  order  to  give  a  speed  "set."  During  the 
pre-test  series,  errors  were  also  emphasized.  When  a  subject  omitted  one  or  more 
target  cards  from  his  sort  the  remaining  deck  was  handed  to  him  with  a  request  that 
he  make  certain  he  had  aU  of  the  targets.  Errors  of  confusion  (including  of  incorrect 
symbols  in  a  sort)  were  immediately  located  by  the  experimenter  using  the  segmented 
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line  code  on  the  back  of  the  cards.  These  errors  were  removed  from  the  sort  and 
called  to  the  subjects'  attention.  Repeat  oorts  were  not  timed.  The  experimenter 
recorded  time  and  both  confusion  and  omission  errors,  and  reported  the  results  to 
the  subject  after  each  series 

This  procedure  in  the  pre -testing  phase  was  intended  to  give  equal  weight  to 
time  and  errors.  On  successive  preliminary  tasks  the  subjects  did  become  more 
efficient  on  both  counts.  It  was  obvious,  however,  tnat  some  were  striving  to  improve 
their  speed  while  others  worked  mainly  on  perfecting  an  error -free  technique.  These 
factors  were  also  balanced  in  the  final  grouping  of  subjects. 


Pre-Test  Ordering 

In  order  to  counterbalance  learning  and  fatigue  effects  on  performance,  a 
double  fatigue  order  was  perserved.  Four  testing  orders  were  followed  in  sequence 
as  subjects  appeared  for  testing  (Table  1). 


TABLE  I 

Testing  Orders  for  Pre  -Test 


Order  1 

Order  2 

Order  3 

Order  4 

1. 

Simple,  numeric 

Simple,  alpha 

Composite,  alpha 

Composite,  numeric 

2. 

Simple,  alpha 

Simple,  numeric 

Composite,  numeric 

Composite,  alpha 

3. 

Composite,  numeric 

Composite,  alpha 

Simple,  alpha 

Simple,  numeric 

4. 

Composite,  aloha 

Composite,  numeric 

Simple,  numeric 

Simple,  alpha 

Presentation  of  the  five  possible  letters  and  the  10  modifieis  also  followed  a 
progressive  rotation  within  each  order.  In  all  instances  the  alternated  decks  followed 
inverse  item  ordering.  This,  cupled  with  rapid  test  sequences,  reduced  the  possibility 
of  learning  the  total  target  counts.  No  subject  stated  that  he  had  noted  a  patterning  in 
card  counts. 

Individual  scoring  sheets  in  the  various  fatigue  orders  held  the  experimenter  to 
his  program  without  difficulty.  To  further  simplify  scoring,  the  correct  count  was 
recorded  in  red  ink  for  each  item  listed.  Score  sheets  were  below  the  subjects’  line 
of  view  at  all  times. 
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Pre-Test  Procedure 


During  the  test  the  subject  and  experimenter  faced  each  other  across  a  table 
In  a  well  lighted,  shadow-free  area.  Each  subject  adjusted  his  chair  until  he  felt 
he  was  handling  a  blank  card  deck  at  his  best.  The  experimenter  then  handed  him 
the  first  test  deck  with  a  cover  card  over  the  symbol.  He  was  instructed  to  keep  ; 

cardc  oriented  with  the  cut  corner  at  th  :  upper  left.  The  experimenter  then  named 
the  required  target  and  at  the  order  "Go"  the  subject  began  his  sort,  slapping  the 
table  with  his  final  card  as  a  signal  of  completion.  Stopwatch  timing  based  on  these 
two  signals  had  proved  most  reliable  in  previous  timing  experiments.  j 

The  experimenter  recorded  the  time  and  moved  the  sorted  cards  to  the  scoring 
section.  He  then  started  the  subject  on  the  target  for  the  second  deck.  There  was 
adequate  time  for  counting  the  sorted  cards  and  checking  confusion  errors  before  the  I  - 

next  sort  was  completed.  If  there  were  no  errors  the  original  deck  was  shuffled  and 
testing  continued.  When  errors  of  omission  were  noted  subjects  were  required  to 
recheck  the  remaining  deck  (untimed).  Errors  of  commission  were  pointed  out  in 
the  sorted  cards.  After  each  complete  test  sequence  there  was  a  two -minute  rest 
period  before  the  following  sequence  began.  Total  testing  and  error -correction 
time  for  an  individual  subject  averaged  about  40  minutes  with  three  rest  periods  i  ■ 

during  the  session. 

Counterbalancing  in  the  later  test  sequences  required  four  matched  groups  of 
five  subjects  each.  During  pre  -testing  seven  subjects  had  been  scheduled  for  each 
testing  order.  If  an  individual  was  highly  irregular  in  his  performance  or  error 
scores  were  excessive,  a  second  subject  was  substituted  in  the  same  sequence.  It 
was  necessary  to  locate  three  additional  subjects  before  matching  the  four  groups  of  j 

five  subjects. 


Results  1 

| 

Matching  of  subjects  was  checked  on  total  testing  time  and  errors.  The  com-  | 

plete  test  run  included  10  double -deck  alphabetic  sorts  and  20  numeric  sorts.  Each 
subject  sorted  3300  cards  and  selected  440  target  cards  during  his  test  session.  i 

Since  there  were  five  subjects  in  each  testing  order,  times  and  errors  within  an 
order  represent  16,  500  cards  with  2200  target  selections.  Table  2  represents  the  f 

adequacy  of  initial  group  matching.  ] 

Counterbalancing  was  adequate  for  the  alpha  versus  numeric  fatigue  ordering 
which  balanced  groups  1  and  4  against  groups  2  and  3.  Summing  across  these  groups  i 

gives  a  time  of  349.28  minutes  with  56  errors  for  subjects  tested  on  numerics  first, 
against  350.71  minutes  and  48  errors  for  the  groups  who  started  on  the  alphabetic 
cards. 
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TABLE  2 


Total  Performance  on  Alphanumeric  Card  Sorts 
N=20 


Order 

N 

Time  in 
Minutes 

Errors 

1 

5 

170.25 

25 

2 

5 

177,48 

37 

3 

5 

173.23 

11 

4 

5 

173.03 

31 

Total 

20 

699.99 

104 

Balancing  on  the  single  and  composite  task  was  less  adequate.  Here  groups  1 
and  2  are  paired  with  3  and  4.  Scores  summed  across  these  pairs  give  a  time  of 
353.73  minutes  and  62  errors  for  the  groups  with  the  simple  sort  initially  against 
346.26  minutes  and  42  errors  for  those  sorting  on  the  composite  task  first.  The 
time  elements  were  reasonably  well  balanced  but  errors  differed  significantly 
between  orders  2  and  3.  Subjects  were  regrouped  to  correct  this  imbalance  on 
later  tests. 

The  pretest  data  was  analyzed  according  to  the  original  groupings  since 
fatigue  orders  entered  into  the  design. 

Statistical  procedures  included  simple  analyses  of  variance,  Neuman-Keuls  (61) 
tests  of  significance  of  differences  and  Pearson  correlations  of  total  times.  Error 
differences  on  total  test  runs  were  tested  by  the  Chi  square  statistic.  Individual  item 
errors  were  too  small  for  this  test. 

On  all  test  runs  variations  between  subjects  exceeded  any  within -subject 
differences.  The  poorest  subject  required  almost  twice  the  time  of  the  best  subject 
on  any  individual  run. 

For  the  simple  alphabetic  sort,  total  times  varied  from  25.27  minutes  for  D 
to  26.02  minutes  for  B.  Differences  were  not  significant.  The  alphabetic  sort  on 
the  composite  deck  varied  from  22.68  minutes  for  A  to  23.73  minutes  for  B  and 
differences  were  again  not  significant. 

Total  times  for  the  complete  alphabetic  runs  were  127.34  minutes  for  the  simple 
task,  against  115.18  minutes  for  the  composite.  The  two  runs  correlated  +.663. 
Student's  (35)  test  for  difference  of  two  correlated  means  gave  a  t  =  2.53  (1  tail  test, 

19  df,  ^99  =  2.528)  with  performance  significantly  superior  on  the  composite  task. 

Errors  (30  on  simple  versus  26  on  composite)  did  not  differ  significantly. 
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On  the  simple  numeric  cards,  total  sorting  time  ranged  from  22.63  minutes 
for  the  blank  to  23.74  for  the  number  five.  Differences  were  not  significant.  On 
the  composite  task  the  performance  ranged  from  21.87  minutes  for  the  numeral 
5  to  23.09  for  the  6  with  differences  again  not  reaching  the  .05  level. 

Total  times  tor  the  complete  numeric  runs  were  233.17  minutes  for  the  simple 
versus  227.44  for  the  composite.  The  two  runs  correlated  +.679.  Time  differences 
did  not  approach  significance  (t-less  than  1.0). 

The  simple  and  composite  tasks  resulted  in  17  and  31  errors  respectively. 

Using  the  chi  square  corrected  for  continuity  (44),  errors  were  significant^  Treater 
on  the  composite  sorts.  In  this  instance  =  3.56  Qi  of  2.71  is  required  ior  P  =  .05). 

Analysis  by  test  order  showed  no  evidence  of  learning  advantages  transferring 
from  alphabetic  to  numeric  runs.  There  was  significant  (.05  level)  time  improvement 
from  the  first  to  third  sorting  task  with  errors  remaining  stable.  On  the  fourth  task 
a  slight  increase  in  errors  and  time  suggested  a  fatigue  factor. 

Reviewing  the  familiarization  data  made  it  appear  that  counting  a  blank  deck 
and  sorting  a  deck  for  an  individual  symbol  required  almost  equal  time.  The  average 
time  for  counting  five  unbalanced  double  decks  (550  cards)  was  6.08  minutes.  Sorting 
the  same  number  of  cards  for  a  single  alphabetic  symbol  averaged  6.37  minutes. 
Correlation  of  the  two  tasks  was  only  +.337.  For  significance  at  the  .05  level  the 
correlation  for  20  subjects  must  be  *.444  (27).  None  of  the  four  sorting  tasks 
correlated  with  the  card  count  at  this  level.  The  highest  correlation  was  slightly 
under  +.370.  Subjects  had  dealt  the  cards  individually  during  counting  procedures. 
During  the  sorting  tasks  all  subjects  had  developed  a  method  of  fanning  or  spreading 
the  cards  and  viewing  them  in  groups  of  three  to  six. 


Conclusions 

The  few  significant  differences  during  the  pretest  were  probably  the  result  of 
poor  grouping  of  subjects.  The  data  was  used  to  regroup  so  that  times  and  errors 
were  essentially  equal  for  the  four  testing  orders  to  be  used  in  the  final  test. 

The  suggestion  of  fatigue  entering  into  the  fourth  testing  order  called  for  longer 
rest  periods  and  shorter  test  sessions  in  the  final  test. 

Correlations  between  card  counting  and  sorting  did  not  reach  signif icance . 
Subjects  handled  the  cards  differently  in  the  two  tasks.  The  improvement  in  perfor¬ 
mance  during  the  first  three  test  runs  may  have  depended  largely  on  changes  in 
sorting  techniques  since  the  alphanumeric  targets  were  highly  familiar.  All  subjects 
had  reached  a  performance  plateau  and  had  settled  down  to  their  preferred  sorting 
method  before  testing  was  completed. 
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The  pre-test  served  its  purpose  by  preparing  subjects  and  experimenters  for 
the  test  proper. 


Sorting  Test 

The  final  symbol  in  isolation  test  followed  the  alphanumeric  design  with  two -deck 
totals  held  constant.  The  suits,  or  basic  dimensions,  were  represented  by  five  geo¬ 
metrical  symbols.  Face  values  or  modifiers  consisted  of  alphanumeric  or  line 
modifiers.  Three  double  decks  of  cards  were  made  up  for  the  test. 

Set  1  consisted  of  the  basic  unmodified  symbols.  Set  2  used  internal  alphanumeric 
modifiers.  Set  3  used  internal  and  external  linear  modifiers  similar  to  the  Naval 
Tactical  Data  System  (NTDS).  Figure  1  illustrates  the  basic  shapes  and  the  linear 
modifiers. 

Geometric  symbols  were  recommended  by  a  contractor  and  were  based  on 
simplicity  and  familiarity,  as  recommended  in  discemibility  studies  abstracted 
from  the  literature.  The  set  included  a  circle,  half -circle  with  inclosing  diameter, 
triangle,  diamond  and  square. 

Size  ratios,  angles  and  linear  modifiers  were  selected  by  the  experimenters 
using  precision  photographic  slides  and  a  back-projection  method  to  simulate  a 
Cathode  Ray  Tube  (CRT).  Symbols  were  first  studied  in  ideal  sharp  detail  using  a 
one -quarter  inch  diaphragm  on  the  short  -throw  projection  lens.  Degradation  effects 
were  simulated  by  increasing  the  diaphragm  opening  (adding  coma  and  spherical 
aberration  to  the  projection  system),  by  decreasing  illumination  and  by  directing 
glare  source  to  produce  specular  reflection  on  the  screen. 

Modifiers  which  distorted  rapidly  were  eliminated  in  this  procedure.  Linear 
modifiers  which  crossed  the  basic  symbol  were  poor  in  the  most  ideal  condition. 
Auxiliary  symbols  based  on  dot  or  line  numerosity  lost  their  coding  value  with  the 
slightest  degradation.  All  modifiers  of  these  two  types  were  removed  from  the  test 
repertoire . 

The  triangle  degraded  more  rapidly  than  any  other  enclosed  figure.  For  this 
reason,  basic  symbol  size  was  determined  by  the  triangle  which  would  enclose  a 
10-point  "B".  Other  symbols  were  ratioed  to  this  basic  size.  Shapes  and  sizes  are 

included  in  Table  3 . 

Symbols  were  drawn  inside  plastic  templates  and  were  centered  within  t  1/4-inch 
radius  on  3  x  5-inch  cards.  A  K&E  61-3270  "KOH-I-NOOR"  Technical  fountain  pen 
with  a  #4  point  gave  a  satisfactorily  constant  line  width  which  averaged  1/9  of  the 
symbol  heights.  Linear  modifiers  were  also  drawn  with  this  pen.  Alphanumeric 
modifiers  used  standard  elite  typewriter  numbers  and  uppercase  letters.  The 
segmented  line  coding  for  error  check  followed  the  pretest  design.  The  corner  cut 
was  continued  as  an  aid  to  proper  orientation. 
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Fig.  1.  FORMS  FOR  SYMBOLIC  CARD  SORT 
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TABLE  3 


Symbol  Specifications  for  the  Sorting  Task 


Form 

Dimension 

Circle 

7/32"  diameter 

Square 

1/4”  side 

Half -Circle 

3/8"  diameter 

Triangle 

90°  isosceles,  1/4"  altitude 

Diamond 

60°  apical  angle,  1/4"  sides 

Subjects  and  Procedure 

The  20  subjects  selected  by  the  pretest  participated  in  the  sorting  task.  The 
four  new  groups  of  five  subjects  each  were  almost  perfectly  matched  on  time  and 
errors  in  the  pretest  sessions. 

Test  administration  and  scoring  methods  followed  the  earlier  design.  In  all 
instances  the  unmodified  test  forms  were  presented  first  as  a  familiarization 
procedure.  The  two  modified  sets  were  presented  in  double  fatigue  order.  Group  1 
and  4  began  with  the  shape  alpha  series,  while  2  and  3  began  with  the  shape  modifier 
series.  The  shape  sort  was  the  first  procedure  for  groups  1  and  2;  the  modifier  sort 
was  first  for  3  and  4. 

Since  the  linear  modifiers  were  unfamiliar  forms,  all  subjects  were  asked  to 
study  the  demonstration  sheet  illustrated  in  Figure  1.  This  material  was  reviewed 
until  a  subject  responded  accurately  when  asked  to  name  an  illustrated  modifier 
or  to  locate  a  named  modifier.  Since  group  3  had  modifiers  of  this  series  as  the 
first  task,  demonstration  and  check  procedures  were  repeated  twice  with  a  30 -minute 
rest  period  intervening.  The  sorting  procedure  followed  the  second  demonstration 
and  a  five -minute  rest.  The  remaining  subjects  were  also  given  two  demonstrations, 
with  the  first  as  a  prelude  to  scheduled  rest  periods  and  the  second  preceding  the 
linear  modifier  sort.  We  could  not  expect  to  equalize  familiarity  of  alphabetic  and 
linear  modifiers  during  these  sessions.  The  demonstrations  did  make  the  material 
familiar  enough  for  some  measurement  of  performance. 
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Results 


Results  closely  approximate  the  pretest.  Table  4  summarizes  times,  standard 
deviations  and  errors  for  the  20  subjects.  Five  item  tests  contained  22  of  each  target 
In  a  double  deck  of  110  cards.  The  times  and  errors  summed  over  all  sorts  for  the 
20  subjects  represent  the  location  of  2200  targets  from  11,  000  cards.  Ten  item  test 
scores  represent  location  of  2200  targets  from  22,  000  cards. 

There  is  a  strong  similarity  in  times  for  the  five  item  sorts.  The  two  simple 
tasks,  alpha  alone  and  shape  alone,  require  more  time  and  result  in  greater  errors 
than  tasks  requiring  location  of  the  same  symbols  from  a  composite  target.  Inter  - 
correlations  of  these  sorts  vary  from  +.67  to  +.93.  Since  a  two-variable,  19  degree 
of  freedom  correlation  coefficient  of  .561  is  significant  at  the  .01  level  (27),  the  five 
item  tasks  are  definitely  related  and  subjects  who  perform  well  on  one  have  a  tendency 
to  perform  well  on  another.  The  five  item  tasks  on  geometric  shapes  all  correlate 
+  .81  or  higher.  The  highest  correlation  (+.93)  is  between  plain  shapes  and  shapes 
complicated  by  a  linear  modifier.  With  this  correlation  the  time  difference 
(Student's  t)  is  significant  at  the  .02  level  for  the  20 -subject  pool. 


TABLE  4 


Summary  of  Card  Sorting  Time  and  Errors,  Total  Runs 

N=20 


Target  Class 

Total  Time 

In  Minutes 

Standard 

Deviation 

Errors  of  Omission 
and  Commission 

Five  Targets  in  110  Cards 

Alphabetic  alone 

127.35 

1.229 

30 

Alpha  of  alphanumeric 

115.19 

1.360 

26 

Shape  alone 

127.75 

2.442 

29 

Shape  of  shape  alpha 

120.16 

1.838 

17 

Shape  of  shape  modifier 

120.62 

1.518 

9 

Ten  Targets  in  110  Cards 

Numeric  alone 

233.17 

4.0843 

17 

Numeric  of  alphanumeric 

225.42 

4.6789 

31 

Alpha  of  shape  alpha 

221.88 

2.733 

13 

Modifier  of  shape  modifier 

227.53 

2.524 

40 
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Plain  shapes  and  the  shapes  of  the  alphanumeric  modified  series  correlate  .818. 
In  this  case,  the  lower  correlation  reduces  the  difference  significance  to  only  .07,  with 
the  time  advantage  for  the  more  complicated  task. 

Simple  analyses  of  variance  of  the  five  targets  in  each  series  and  Newman-Keuls 
tests  on  differences  between  targets  give  little  information.  In  all  cases,  the  signifi¬ 
cant  variance  is  between  subjects.  The  highest  within -treatment  or  intersymbol 
F  ratio  was  1.09  (F  „  -  (4,  76)  =  2.51).  In  no  series  does  an  individual  target  difference 
approach  the  .05  level  of  significance.  Total  times  for  the  three  forms  of  modified 
targets  again  reveal  no  significance  difference . 

Errors  were  greatest  on  the  plain  shape  test.  A  chi  square  treatment  gives  a 
value  of  11.0,  which  is  significant  at  the  .01  level  (df  =  2).  Since  this  was  the  first 
sort  in  all  instances,  it  was  probably  at  a  disadvantage.  Errors  related  to  the  two 
forms  of  modification  give  a  of  1.89.  Since  2.71  is  the  value  required  for  a  .05 
significance  (df  =  l),  the  data  does  not  indicate  an  advantage  to  either  method.  While 
confusion  and  omission  errors  were  consistently  most  frequent  on  triangles  and 
diamonds,  the  number  of  errors  was  too  small  for  chi  square  testing. 

Similar  treatment  of  modifier  da.  gives  a  slight  but  not  significant  time 
advantage  to  the  alphanumeric  series.  E  rror  differences  gave  a  value  of  12.74, 
indicating  that  the  alphanumeric  series  re  salts  in  significantly  (.01  level,  df  =  1) 
less  errors.  This  advantage  is  of  questionable  value  since  subjects  were  much  less 
familiar  with  the  linear  modifiers. 

Analysis  of  variance  and  Newman-Keuls  tests  reveal  no  significant  differences 
within  modifiers  of  a  series.  Errors  were  spread  quite  evenly  over  the  various 
modifiers. 

Correlations  of  all  sorting  times  with  the  card  count  task  continue  at  +  .32 
to  +.38.  Correlations  with  the  similar  pretest  scores  are  all  between  +.75  and 
+  .85.  Time  and  error  differences  between  similar  tests  are  not  significant. 


Conclusions 

When  the  card  sorting  technique  is  used,  the  symbols  recommended  by  the 
contractor  appear  almost  equally  salient.  Since  the  sizes  of  the  five  targets  were 
ratioed  to  give  equal  discemibility,  the  equality  findings  encourage  dependence  on 
pure  legibility  or  visibility  advantages  for  the  selection  of  symbols.  The  testing 
method  must  be  validated  against  a  more  realistic  task  simulation  before  such  a 
selection  procedure  can  be  recommended. 

Sorting  procedures  require  visual  alertness  and  dexterity  in  handling  cards. 
Since  the  radar  operator's  task  also  involves  both  perceptual  and  motor  skills,  there 
is  some  similarity  between  the  test  and  the  desired  skills.  While  sorting  in  a  single 
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dimension  required  almost  the  same  time  as  counting  a  card  deck  of  equal  size,  the 
two  measures  correlated  at  a  very  low  level  and  offered  no  solution  to  separating  the 
perceptual  and  the  motor  portions  of  the  task. 

There  was  some  possibility  of  determining  task  breakdown  within  the  test 
design.  Each  basic  symbol  appeared  in  20  percent  of  the  total  decks,  while  each 
modifier  was  present  in  only  10  percent.  To  locate  an  equal  number  of  modifiers, 
subjects  handled  twice  the  number  of  cards  used  in  the  basic  symbol  sorts.  The 
differences  in  time  on  the  five  and  10-item  tasks  could  provide  a  measure  of  the 
motor  portion  of  the  task. 

Similarly  half  of  the  time  for  modifier  sorting  required  total  card  handling 
equal  to  the  entire  alphabetic  sort.  In  this  case,  the  target  cards  were  also  reduced 
by  half  and  time  difference  might  result  from  having  fewer  targets  from  the  decks. 

The  three  composite  sorts  for  20  subjects  were  used  to  estimate  time  factors. 
Results  indicate  an  average  time  of  .580  seconds  to  reject  a  card  and  .915  seconds 
to  accept  it.  The  first  figure  might  be  considered  as  the  time  for  manipulation, 
recognition  and  a  "no-target"  decision.  The  additional  .335  seconds  could  represent 
confirmation  of  a  "yes"  decision  and  separation  of  the  target  card  from  the  deck. 

These  figures  hold  up  well  enough  in  predicting  sorting  on  the  geometric  forms 
that  there  may  be  some  meaning  to  them.  They  also  suggest  that  card-sorting 
procedures  may  consist  of  a  series  of  binary  "yes -no"  decisions  with  a  verification 
of  all  "yes"  decisions.  Under  such  conditions,  the  testing  procedure  does  not  appear 
to  be  closely  related  to  the  radar  operator's  task. 

If  the  card  sorts  are  a  series  of  individual  decisions,  symbol  recognition  may 
be  a  minute  fraction  of  reaction  time.  In  this  case,  findings  may  be  of  little  value 
for  our  use.  The  following  experiment  is  an  effort  to  validate  the  sorting  procedure 
as  a  measure  of  target  visibility  on  a  mixed  display. 
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EXPERIMENT  Q  -  CONFUSION  STUDIES  -  FIRST  SERIES 


While  the  sorting  techniques  of  experiment  I  suggests  that  the  recommended 
symbols  may  be  well  chosen,  the  test  In  no  way  represented  the  radar  operator's 
task.  The  radar  scope  presents  information  on  a  number  of  targets  of  mixed  types 
scattered  across  the  scope  as  a  poorly  organized  array.  The  operator  decodes  this 
array  and  acts  upon  it. 

During  surveillance  periods  friendly  air/supported  targets  (AST)  flying  anticipated 
paths  dominate  the  screen.  An  occasional  unknown  or  unidentified  craft  appears, 
usually  entering  at  the  perimeter.  The  operator  watches  for  changes  in  flight 
patterns,  or  deviations  from  air  corridors.  When  an  unknown  target  enters  the 
screen  he  interrogates  it  and  awaits  identification  as  he  follows  its  course.  At  this 
stage  his  task  is  monitoring  and  alerting  with  only  two  basic  target  types  on  his 
screen. 

During  an  engagement,  the  screen  presents  a  rapidly  shifting  picture.  Unknown 
symbols  become  friendly  or  hostile  as  identification  is  completed.  The  unidentified 
category  is  now  potentially  hostile  and  requires  rapid  identification.  Enemy  AST  taking 
advantage  of  terrain  contours  or  cloud  formations  may  have  penetrated  deep  into  the 
areas  under  surveillance,  before  appearing  as  "pop-up"  targets.  Hostile  tactical 
ballistic  missiles  (TBM)  may  appear  suddenly  and  our  own  TBMs  come  into  play  as 
needed.  The  operator  is  now  faced  with  five  target  classes  of  variable  densities. 

At  this  point  his  task  has  become  more  complicated  and  his  performance  is  a  critical 
factor  in  the  success  of  defensive  or  retaliatory  action.  It  is  in  this  situation  that  an 
ideal  coding  method  may  create  a  temporal  and  informational  advantage  leading  to 
increased  mission  effectiveness. 

It  is  improbable  that  the  radar  operator  who  faces  such  a  crowded  and 
heterogeneous  picture  will  study  each  symbol  individually  and  make  a  binary  decision 
on  its  representational  value.  For  efficient  decision-making  he  needs  to  see  the 
entire  engagement  picture  as  a  background  with  targets  so  chosen  that  individual 
classes  may  be  called  into  a  patterned  figure.  Ideally,  the  figure  ground  effect 
should  be  immediately  reversible  with  target  classes  called  into  saliency  at  will. 

Such  selectivity  may  be  accomplished  by  push-button  callup;  however,  this  would 
Increase  operator  actions  within  a  critical  time  period. 

A  highly  discriminable  basic  coding  alphabet  with  a  perceptual  organization 
hierarchy,  or  multiple  figure  -ground  reversibility,  would  be  ideal.  Since  the  radar 
scope  presents  an  irregularly  changing  scene,  the  Gestalt  figure  -ground  rules  of 
proximity,  closure,  continuity  and  common  fate,  cannot  be  forced  into  the  situation. 
What  seems  to  be  needed  is  a  family  of  highly  discernable  symbols  differing  so 
strongly  along  some  major  dimension  that  each  class  may  be  individually  perceived 
as  the  figure. 
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Rules  for  a  good  geometric  shape  coding  have  been  repeatedly  listed: 

a.  Primary  symbols  should  be  large  and  enclose  a  space. 

b.  No  auxiliary  symbol  should  cross,  distort,  interfere  with  or  in  any  way 
obscure  the  primary  symbol(s). 

c.  Symbol  complines  should  not  normally  exceed  two  geometric  symbols 
(possibly  three  in  some  circumstances),  a  location  dot,  and  a  speed  and  direction 

vector  line  where  applicable.  ; 

d.  When  other  information  is  required,  it  should  be  in  the  form  of  numerical 
representation  (e.g. ,  one,  two,  or  three  marks  to  indicate  magnitude  of  object)  or 

in  actual  numbers  and  letters.  1 

e.  The  geometric  center  of  the  symbol  and/or  a  large  clear  dot  should  indicate 
location. 

f.  Auxiliary  marks  should  be  compact  solid  figures.  ; 

The  original  source  of  this  series  of  rules  is  not  referenced  and  has  not  been 
located.  Generally,  they  are  supposedly  based  on  Gestalt  logic  or  "good -figure'1 
design  for  individual  symbols.  Bowen  (13)  points  out  that  the  Gestalt  rules  tend  to 
produce  similarity  of  shape  while  discrimination  calls  for  non-similar  forms.  ; 

Bowen's  studies,  however,  continue  to  follow  the  rules  given  above,  probably  because  I 

it  is  almost  impossible  to  produce  a  series  of  forms  without  a  degree  of  similarity. 

The  forms  recommended  for  our  basic  studies  and  used  in  the  card -sorting 
experiment  conform  to  the  listed  ru.es  and  were  probably  selected  on  that  basis.  j  • 

While  they  appear  to  be  well  selected  for  isolation  studies,  the  question  is  whether 
they  will  be  equally  satisfactory  as  a  mixed  display. 

1 

Test  Design  j 

Confusion  studies  were  designed  to  approximate  the  anticipated  radar -operator 
task  at  a  moderate  saturation  level.  A  north -oriented,  90°  sector  was  plotted  on 
8  x  10  1/2 -inch  white  paper.  The  x,  y  coordinates  for  the  25  target  positions  as 
illustrated  in  Figure  2  were  located  by  a  random -numbers  technique.  All  target  i 

centering  held  within  a  *  1/8 -inch  radius  of  the  chosen  points.  j 

Individual  test  sheets  contained  a  set  of  five  target  symbols  with  each  symbol 
appearing  four,  five  or  six  times.  A  test  run  consisted  of  10  different  test  sheets 
with  varying  symbol  locations.  Within  a  test  run  each  symbol  appeared  twice  in 
each  position  (for  a  total  of  50  times  within  a  test  run).  This  design  equalized  search 
areas  for  the  various  symbols  without  creating  a  strong  patterning  effect. 
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Fig.  2.  LOCATION  OF  TARGETS  IN  CONFUSION  STUDIES 
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Master  test  sheets  were  hand  drawn  with  a  BIG  broadline,  black  ink,  ballpoint 
pen.  Line  drawings  with  this  pen  were  equal  in  width  to  the  #4  Rapidograph  but 
inking  was  more  irregular  for  better  simulation  of  CRT  reso’  .1.  Plastic  templates 
were  used  to  maintain  symbol-size  regularity. 

The  first  test  series  utilized  the  contractor -recommended  symbols  of  the 
isolation  test.  An  equilateral  triangle  was  substituted  at  the  suggestion  of  subjects. 
This  series  included  four  separate  sets  of  material:  (a)  unmodified  symbols, 

(b)  symbols  modified  by  uppercase  letters  within  the  symbol,  (c)  symbols  modified 
by  uppercase  letters  exterior  and  to  the  right  of  the  symbol,  and  (d)  symbols  with 
linear  NTDS  modifiers.  All  modifiers  were  included  to  measure  their  interference 
with  shape  recognition  and  to  increase  clutter.  Since  the  modifiers  did  not  have  to 
be  identified  in  testing,  they  were  randomly  placed. 

The  basic  symbols  are  described  in  Table  5.  The  four  complex  display  sets  of 
this  series  are  illustrated  in  Figures  3  through  6. 


TABLE  5 

Basic  Symbols  for  Series  1 


Meaning 

Shape 

Size 

Friendly,  AST 

Circle 

7/32 -inch  diameter 

Friendly,  TBM 

Half  circle  with 
enclosing  diameter 

3/8 -inch  diameter 

Hostile,  AST 

60°,  120°  diamond 

3/ 16 -inch  sides 

Hostile,  TBM 

Equilateral  triangle 

1/4-inch  sides 

Unknown 

Square 

1/4-inch  diagonal 

Ti  e  final  test  sheets  were  fifth -generation  xerox  copies,  since  optimal  contrast 
and  resolution  is  not  to  be  expected  in  any  changing  display.  Kleer-Vu  plastic  covers 
provided  protection,  an  erasable  field  and  a  surface  for  a  realistic  specular  reflection 
common  to  CRTs. 

A  full  test  sequence  required  five  copies  (one  for  each  basic  symbol  tested)  of 
each  10-sheet  run.  Randomized  order  within  runs  of  10  further  reduced  recognition 
of  patterning  effects  during  testing. 
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Fig.  3.  SERIES  la 
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Each  50 -sheet  sequence  for  a  symbol  set  was  mounted  in  loose-leaf,  hard-back 
notebooks  for  rapid  presentation.  The  subject  marked  the  designated  targets  with  a 
pencil-type  felt  marking  pen  as  he  turned  through  10  test  sheets  to  complete  his  run. 
The  markings  could  be  removed  with  an  alcohol-saturated  cloth  after  scoring.  The 
plastic  covers  cleaned  completely  without  stain  and  the  materials  remained  in 
excellent  condition  throughout  the  period  of  testing. 


Subjects 

The  subjects  for  the  first  series  were  the  same  used  In  the  symbols -in -isolation 
study.  Regrouping  produced  almost  perfect  balance  on  time  and  errors  on  that  test 
to  maintain  fatigue  order.  All  these  subjects  had  considerable  familiarity  with  the 
first  series  as  a  result  of  the  previous  tests.  Their  cooperation  and  interest  were 
excellent. 


Procedure 

A  large  well-lighted  table  was  used  for  all  confusion  tests.  A  Weston  Model  614 
Foot -Candle  Meter  gave  an  average  reading  of  62  foot -candles  on  the  table  surface. 
The  task -area  brightness,  on  the  open  test  book,  averaged  35  foot-lamberts  as  mea¬ 
sured  by  a  Photo  Research  Corp.  Spot  Brightness  Meter,  Model  SB  1/2,  Immediate- 
surround  brightness  at  the  table  averaged  10  foot-lamberts,  giving  a  task -to -surround 
brightness  ratio  of  3.5:1  and  a  task -to -surround  contrast  of  71  percent.  Specular 
glare  areas  on  the  test -sheet  covers  were  60  foot-lamberts  or  more. 

The  subject  chose  a  chair  of  comfortable  height  and  readjusted  it  to  his  satis  - 
faction.  The  experimenter  presented  a  notebook  of  practice  sheets  similar  to  the 
test  material  and  explained  the  task.  Each  subject  was  instructed  to  turn  the  cover 
sheet  at  the  word  "Go,"  to  mark  all  of  the  circles  on  each  successive  sheet,  and  to 
tap  his  pen  as  he  finished  the  final  sheet. 

The  experimenter  timed  each  run  and  observed  any  hesitations  in  starting  or 
in  signaling  the  end.  If  a  subject  hesitated  in  signaling,  the  experimenter  designated 
a  second  symbol  as  the  target  in  a  repeat  run.  Subjects  cleaned  the  practice  sheets 
between  runs  while  the  experimenter  discussed  the  similarity  of  the  test  to  the  tasks 
of  a  radar  operator.  After  a  subject  had  stabilized  in  his  signaling  procedures,  he 
was  given  a  two -minute  rest  period  before  the  test  proper. 

Each  of  the  subject  groups  was  assigned  one  of  the  four  ordered  permutations 
of  sub -test  modif  ications .  The  five  basic  symbols  were  similarly  ordered  within  the 
five  subjects  of  each  group.  Fatigue  and  learning  factors  were  thus  counterbalanced 
both  within  and  betv'een  tests. 

Each  subject  completed  all  five  targets  of  a  set  in  rapid  succession,  pausing 
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only  until  the  experimenter  recorded  time  and  designated  the  next  symbol.  When  he 
completed  his  first  set,  the  subject  was  given  a  rest  break  and  asked  to  walk  around 
and  stretch  his  legs.  During  this  period  the  experimenter  iocated  and  recorded 
errors  of  confusion  or  omiseion. 

When  the  subject  returned  he  was  told  how  many  errors  he  made.  Times  were 
given  only  on  request,  since  subjects  regularly  looked  at  the  stopwatch  as  they  tapped 
the  table  at  the  end  of  a  run.  The  next  set  of  the  appropriate  order  was  imn  tjiateiy 
introduced.  Testing  again  included  the  five  symbols  of  the  set  in  rapid  succession. 
The  subject  then  cleaned  his  first  test  book  while  the  experimenter  scored  errors 
on  the  second  set. 

The  same  procedure  was  followed  until  the  fourth  set  was  completed.  During 
the  final  scoring  period  subjects  were  asked  to  fill  in  a  questionnaire  (see  Appendix  A) 
regarding  the  test. 

Results 

A  simple  analysis  of  variance  considering  the  five  symbols  as  the  repeated 
measures,  or  treatment,  again  seemed  the  most  direct  method  for  studying  the  data. 
This  design  would  also  provide  the  basis  for  defining  significant  time  differences  by 
the  Newman-Keuls  method  (61).  The  statistical  procedure  pattern  is  included  in 
Appendix  B,  Table  IB.  Raw -data  and  total -data  reduction  figures  have  been  omitted 
for  simplicity.  All  data  sheets  are  on  file  at  the  Human  Engineering  Laboratories. 

In  the  first  procedure,  each  set  of  the  series  is  considered  separately  to  point 
out  symbol  trends.  A  second  analysis  uses  total  time  on  each  of  the  four  complete 
sequences  to  locate  modifier  advantages.  Chi  square  analysis  of  error  trends  is 
treated  separately.  All  sets  of  Series  I  demonstrate  homogeneity  of  variance  by 
the  Fmax  test.  (61).  Analysis  of  variance  reveal  significant  differences  at  the  .01 
level  for  both  subjects  and  symbols. 

Table  6  presents  a  summary  of  times  and  errors  for  the  20  subjects  used  in 
this  series.  In  this  table  a  run  represents  location  of  50  symbols  from  a  10 -page 
presentation  including  250  mixed  symbols.  A  sequence  represents  the  five  symbol 
runs  of  a  set  with  250  symbols  located  in  a  total  exposure  of  1250. 

The  earlier  card -sorting  test  using  the  symbols  of  this  series  had  indicated 
equality  of  search  time  for  the  different  geometric  forms.  When  the  same  symbols 
were  located  on  a  mixed  display,  this  equality  vanished.  In  all  four  sets  of  this 
series  the  circle  and  half -circle  targets  were  located  in  at  least  25  percent  less 
time  than  triangles  and  diamonds.  Errors  differed  even  more  greatly. 
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TABLE  6 


Summary  of  Times  and  Errors  on  Experiment  II 
N=20 


_ _ Shape _ 

Triangle  Circle  Diamond  Square  Half  Circle  Total 


Set 


a.  Unmodified 


T 

22.40 

14.71 

22.85 

21.46 

15.24 

96.66 

s2 

.0566 

0435 

.0702 

.0399 

.0339 

.0794 

E 

38 

i 

34 

25 

7 

105 

b.  Inner  alpha 

T 

24.87 

19.06 

26.14 

24.63 

16.67 

111.37 

s2 

.0472 

.0476 

.0697 

.0588 

.0524 

.0877 

E 

31 

7 

32 

6 

2 

78 

c.  Outer  alpha 

T, 

23.25 

17.13 

24.33 

22.05 

16.63 

103.39 

s2 

.0531 

.0475 

.0562 

.0683 

.0650 

.0814 

E 

23 

4 

33 

6 

3 

69 

d.  Line  Modifier 

T 

23.37 

15.76 

25.16 

20.94 

15.83 

101.06 

82 

.0585 

.0353 

.0847 

.0485 

.0462 

.0862 

E 

22 

0 

27 

12 

8 

69 

s 

E 


=  Sum  of  time  in  minutes  for  20  subjects  for  each  run  or  sequence 
=  Variance 

errors  of  omission 


Sets  of  Series  I  are  illustrated  in  Figures  3  through  6. 
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Symbols  within  the  sets  differed  significantly.  According  to  the  Newman -Keul 
statistic,  the  circle  and  half -circle  are  superior  to  the  linear  figures  at  the  .01  level 
in  all  sets.  The  square  is  the  best  of  the  linear  forms  and  is  superior  to  the  triangle 
and  diamond  at  the  .05  level. 

Differences  in  errors  are  significant  at  the  .01  level,  according  to  the  chi 
square  statistic.  The  diamond  and  triangle  account  for  75  percent  of  all  errors  of 
omission.  There  were  only  22  errors  of  confusion  in  the  entire  series.  All  but  one 
of  these  was  between  the  diamond  and  the  triangle. 

Total  times  for  a  complete  sequence  of  the  five  symbols  within  a  set  are  also 
treated  by  analyses  of  variance.  Subject  and  treatment  variances  are  both  significant 
at  the  .01  level.  The  data  is  homogeneous  in  terms  of  the  F  ,  test.  Newman-Keul 
tests  of  difference  indicate  that  the  times  for  the  unmodified  ?orrns  are  superior  to 
the  linear  modifiers  at  the  .05  level  and  to  the  two  alphabetic  sets  at  the  .01  level. 
Linear  modifiers  or  letters  to  the  right  of  the  symbols  do  not  differ  significantly  In 
their  effect  on  total  time  and  are  both  superior  at  the  .01  level  to  letters  within  the 
symbols.  While  differences  are  significant,  they  represent  time  savings  of  only 
about  10  percent. 

The  unmodified  forms  of  set  1  were  processed  most  rapidly;  however,  errors 
were  greatest  on  this  set.  This  suggests  that  modifying  information  on  the  screen 
may  keep  the  operators  more  alert. 

Linear  modifiers  had  little  effect  on  the  times  for  the  circle,  half -circle  and 
square.  The  use  of  alphabetic  modifiers  within  the  symbol  increased  times  for  all 
forms  and  also  gave  the  highest  error  count  of  any  of  the  modified  sets. 

Subjects  had  a  tendency  to  be  fast  or  slow  consistently  regardless  of  symbol  or 
set.  Table  7  records  a  select  series  of  Pearson  Product  Moment  correlations  which 
illustrate  this  trend.  Proper  selection  of  operators  could  significantly  reduce  search 
time.  Our  fastest  subject  processed  data  in  approximately  one -half  the  time  of  our 
slowest  throughout  the  test. 

It  is  especially  interesting  that  neither  card -sorting  of  symbols  nor  card 
counting  correlates  with  the  task  of  locating  symbols  in  a  mixed  array. 
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Task 

Mean 

r 

Index  of 

Forecasting  Efficiency 

1.  Series  Id  circle  vs 

.788 

.777  ** 

.3705 

Series  Id  triangle 

1.258 

2.  Total  sequence  la 

4.833 

.9305  ** 

.6339 

vs  Total  sequence  lb 

5.568 

3.  Total  sequence  Id 

5.053 

-.005 

.00001 

vs  Shape  of  shape 

6.031 

modifier  card  sort 

4.  Total  sequence  Id 

5.053 

-.056 

.0016 

vs  Blank  card  count 

.608 

5.  Card  sort  plain  shape 

6.392 

.387 

.078 

vs  Blank  card  count 

.608 

**  .01  significance  level,  N  =  20  requires  r  =  .561  (27) 


Discussion 

The  confusion  tests  present  information  in  a  reasonable  approximation  to  the 
anticipated  PPI  display.  Since  there  is  essentially  zero  correlation  between  per¬ 
formances  on  isolated  symbols  and  mixed  arrays,  these  experiments  do  not  Justify 
the  use  of  sorting  tests  to  select  ideal  shapes  for  a  geometric  code. 

In  both  experiments  symbol  sizes  were  ratioed  to  relative  visibility.  Sorting 
tests  indicated  that  the  design  produced  equal  salience.  That  confusion  studies  did 
not  give  similar  results  suggests  that  pure  discernlbility  is  inadequate  as  a  lone 
criterion  in  selecting  a  code.  There  appears  to  be  interference  or  competition  among 
the  various  straight-line  forms  when  presented  in  a  mixed  array.  Discrim  inability 
and  discernlbility  are  not  identical  factors. 


The  five  basic  symbols  selected  for  study  differ  significantly  in  discriminability. 

The  circle  and  half -circle  are  located  rapidly  with  few  errors  of  omission  or  commission. 


The  triangle  and  diamond,  while  equally  visible,  require  almost  50  percent  more 
time  than  the  curved  figures  and  account  for  eight  times  the  errors.  The  square 
gives  an  intermediate  performance . 

If  these  were  the  only  symbols  or  modification  methods  available,  the  choice 
would  lie  between  linear  or  outer  alpha  modifiers.  The  linear  methods  would  probably 
be  most  promising  because  of  their  great  flexibility  in  coding.  The  greatest  time-  j 

saving  possibility  would  then  be  realized  by  proper  operator  selection. 

Unfortunately  we  have  no  clues  on  operator  selection.  Examination  of  the  data 
shows  no  trends  relative  to  age,  education,  visual  acuity  or  interests.  The  two 
fastest  subjects  differed  in  age,  education  and  interests  and  were  comparable  only 
in  their  great  boredom  with  the  test.  The  two  slowest  subjects  were  similarly  ill-  L  . 

matched  except  in  performance. 

The  subject  with  lowest  visual  acuity  was  consistently  superior  in  time  and  j 

error -free  performance  throughout  this  series  and  a  number  of  subsequent  tests.  ■ 

He  was  a  young  enlisted  man  and  a  college  graduate  in  business  administration.  j 

Another  young  enlisted  man  who  had  excellent  performance  in  this  and  subsequent  1 

tests  was  also  a  business  administration  major.  Perhaps  these  subjects  had  gained 

skill  in  working  with  sheets  of  figures.  While  this  might  be  a  highly  selective  j 

criterion,  it  is  not  a  usable  selective  method  for  radar  operators,  who  are  chosen 

with  an  eye  to  their  technical  ability  to  maintain  the  electronic  components.  \ 

Operator  selection  methods  should  probably  be  given  additional  consideration 
in  an  effort  to  maximize  performance .  Scanning  methods  used  by  the  best  operators 
should  also  be  reviewed,  since  training  procedures  provide  another  avenue  toward 
improved  efficiency. 

s  « 

While  operator  selection  and  training  deserve  additional  investigation,  proper 
selection  of  symbols  presents  the  most  promising  approach  to  rapid  and  accurate 
identification  of  targets.  Performance  on  the  various  forms  was  highly  correlated, 
with  strong  differences  in  time  and  errors.  These  differences  spread  across  the  i; 

subject  population  without  exception.  In  such  a  situation,  an  ideal  choice  of  symbols 
within  a  set  could  be  expected  to  increase  the  efficiency  of  any  operator.  Our  efforts  j 

were  directed  toward  this  objective.  I 
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EXPERIMENT  III 


Symbol  Meaning 

Previous  studies  (22)  in  these  laboratories  covered  68  geometrical  forms 
representative  of  symbols  which  might  be  generated  on  a  CRT  by  current  equipment. 
Symbols  were  drawn  on  individual  cards.  Subjects  sorted  the  cards  into  friendly, 
hostile  and  unknown  categories  and  then  ranked  their  choices  within  each  category. 

In  order  to  avoid  an  absolute  forced  choice,  a  fourth  category,  "other,  "  was  permitted. 
Subjects  were  told  that  only  symbols  with  no  meaning  or  with  a  strong  non -military 
meaning  should  be  placed  in  this  category.  When  cards  were  selected  as  "other,  " 
the  subjects  designated  the  meaning  by  a  note. 

Results  indicated  that  all  simple  enclosed  forms  had  a  generally  friendly 
meaning.  Of  these  only  the  circle  was  ranked  consistently  high  within  that  meaning. 

As  internal  linear  modifiers  were  added  to  these  forms,  the  friendly  meaning  weakened. 
As  the  modifiers  increased,  the  form  became  increasingly  hostile.  External  linear 
modifiers  shifted  all  symbols  into  the  unknown  or  into  the  "other"  category  where  they 
were  identified  as  either  too  complicated  or  without  meaning.  We  strongly  suspected 
that,  in  spite  of  instructions,  the  subjects  had  interpreted  the  unknown  category  as 
"I  haven't  the  vaguest  notion." 

The  friendly  connotation  of  all  simple  enclosed  forms  and  changes  in  basic 
meaning  with  addition  of  modifiers  did  not  suggest  that  the  NTDS  system  extension 
used  in  Experiments  I  and  II  were  ideal.  Difficulties  in  locating  the  triangle,  square 
and  diamond  in  Experiment  II  also  indicated  that  these  symbols  lacked  the  definite 
qualities  needed  in  a  useful  code. 

Looking  over  the  results  of  our  earlier  stereotyped  meaning  studies  we  were 
struck  with  the  fact  that  four  of  the  five  symbols  in  Experiment  I  and  II  had  friendly 
meanings.  The  half -circle,  concave  downward,  had  not  been  selective  in  the  meaning 
test  but  had  excelled  in  the  simple  learning  experiment.  Of  the  four  friendly  symbols 
the  circle  had  been  ranked  significantly  highest  and  had  been  superior  in  the  learning 
experiment.  In  fact,  the  circle,  square,  triangle  and  diamond  held  the  same  positions 
in  the  simple  learning  study  as  in  Exper’ment  II.  Perhaps  there  was  a  bit  more  meat 
in  our  earlier  findings  than  we  had  realized. 

The  original  battery  of  68  symbols  had  been  complicated  by  modifiers,  forms 
differing  only  in  orientation  or  connectivity  and  other  strong  deviations  from  simplicity. 
It  seemed  possible  that  the  task  had  been  too  complicated  for  clear-cut  decisions. 

We  set  about  to  reduce  this  battery  to  its  basic  form.  All  modified  figures  were 
deleted.  Compound  forms  which  were  continuous  enclosures  made  up  of  two  basic 
symbols  were  reduced  to  a  single  "ice  cream  cone.”  This  form  had  shown  up  well 
in  the  earlier  studies  and  was  retained  in  both  orientations.  Certain  forms  such  as 
the  asterisk  would  degrade  beyond  recognition  on  the  CRT  .  These  also  were  dr  opped. 
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The  Saint  Andrew's  cross,  the  snow  flake  and  the  crossed  ovals  (or  atom)  would 
approach  each  other  In  form  at  reduced  size.  Of  these,  the  atom  was  retained 
since  it  is  the  simplest  to  generate. 

All  straight-line  enclosed  forms  and  their  parts  had  been  duplicated  by  similar 
curved  line  counterparts.  This  confusion  appeared  poorly  founded  and  the  distorted 
forms  were  omitted.  Eventually  the  battery  of  symbols  was  reduced  to  23. 

This  reduced  battery  was  drawn  up  into  a  format  including  the  253  possible 
pairs.  Each  symbol  appeared  first  in  11  pairs  and  last  in  the  remaining  11.  The 
ordering  of  pairs  was  randomized.  The  material  was  arranged  on  3  x  11 -inch  lined 
sheets  and  the  10  sheets  were  xeroxed.  The  forms  are  included  In  Table  8. 


Subjects 

The  20  subjects  from  Experiment  I  ana  II  were  given  the  paired -comparison 
meaning  test.  All  had  completed  questionnaires  (Appendix  A)  prior  to  testing. 
Subjects  were  tested  in  pairs,  but  they  were  on  opposite  sides  of  the  room  and  seated 
back  to  back.  An  experimenter  remained  in  the  room  during  the  test. 


Procedure 

Subjects  had  become  familiar  with  the  problems  of  the  radar  operator  in 
previous  instructions.  They  were  given  further  information  on  the  use  of  coded 
data  in  the  air  defense  picture.  Friendly,  hostile  and  unknown  meanings  were 
explained  in  patterned  random  order  to  each  group  of  subjects.  After  each  meaning 
had  been  defined,  subjects  were  given  a  set  of  the  comparison  sheets  and  a  3  x  5 -inch 
card.  The  experimenter  explained  that  the  symbols  on  the  forms  were  to  be  consid¬ 
ered  one  line  at  a  time.  The  card  was  to  be  used  to  follow  the  material  rapidly  by 
sliding  it  down  the  page  one  line  at  a  time.  The  subject  was  asked  to  consider  each 
pair  as  the  only  symbols  available  for  the  designated  meaning.  He  was  to  circle  the 
symbol  he  would  prefer  in  that  choice.  Instructions  followed  a  paired -comparison 
routine  of  emphasizing  selection  without  prolonged  thought.  Selection  was  continuous 
and  uninterrupted. 

After  completing  one  meaning  selection  the  subject  was  given  a  second  series 
of  sheets  containing  identical  material  in  a  different  order.  He  was  told  that  these 
were  the  same  symbols  but  that  he  was  now  to  consider  their  use  for  a  second  mean¬ 
ing.  The  third  meaning  was  treated  in  the  same  manner. 

Data  was  then  tabulated  for  the  total  number  of  times  each  symbol  was  circled 
for  an  Individual  meaning.  Since  there  were  20  subjects,  a  symbol  could  be  circled 
a  maximum  of  440  times  in  any  meaning  category. 
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TABLE  8 


Symbology  Meaning 
Faired  Comparisons  -  Totals 
N  =  20 


FRIEND  FOE  UNKNOWN 


No  of  Choices 

Rank 

No  of  Choices 

Rank 

No  of  Choices 

Rank 

GRAND  TOTAL 

241 

9.5 

189 

16 

211 

15 

641 

131 

22 

285 

4 

201 

17.5 

617 

189 

17 

298 

2 

2  19 

5 

736 

213 

14 

240 

10 

212 

13.5 

665 

224 

12 

248 

7 

213 

12 

685 

148 

21 

293 

3 

210 

16 

651 

*2 

8 

178 

18 

219 

10 

639 

250 

5 

242 

9 

230 

9 

722 

241 

9  5 

184 

17 

294 

2 

719 

206 

16 

201 

14 

263 

4 

670 

290 

2 

145 

22 

212 

13  5 

647 

163 

20 

250 

6 

237 

7  5 

650 

243 

7 

175 

19 

237 

7.5 

655 

173 

IV 

261 

5 

295 

1 

729 

357 

1 

66 

23 

130 

23 

553 

266 

4 

198 

15 

201 

17.5 

665 

121 

23 

306 

1 

216 

11 

643 

247 

6 

246 

8 

279 

3 

772 

211 

15 

236 

12 

163 

21 

610 

269 

3 

152 

21 

162 

22 

583 

237 

1 1 

166 

20 

243 

6 

646 

217 

13 

237 

11 

170 

20 

624 

185 

18 

234 

13 

195 

19 

614 
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Results 
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Results  are  tabulated  on  Table  8.  The  total  occasions  each  symbol  was  preferred 
for  a  given  meaning  is  recorded  for  each  symbol.  Ranking  of  symbols  within  each 
meaning  category  is  also  included. 

The  circle,  oval,  and  square  are  irequenlly  selected  as  friend  and  at  the  same 
time  rank  very  definitely  low  in  the  unknown  and  hostile  meanings.  A  rectangle  with 
the  shorter  dimension  vertically  oriented  ranks  high  as  a  friend  but  has  low  selectivity 
in  meaning.  Beyond  this  rank  there  is  no  clear-cut  break  between  ranking  as  friend 
or  net  friend. 

Within  the  hostile  category  the  first  two  choices  also  rank  quite  high  as  unknown. 
The  jet  plane  and  the  half  square  open  at  the  bottom  must  be  considered  simply  as  not 
friendly.  The  third  and  fourth  ranking  forms  in  the  hostile  meaning  are  the  vees, 
apex  up  and  down.  These  symbols  are  highly  selective  for  the  hostile  meaning.  It 
is  interesting  that  the  triangles,  which  are  merely  completions  of  the  vees  do  not 
fo*low  any  meaning  trend. 

The  unknown  category  is  again  the  least  clearly  defined.  The  intersecting  arcs 
show  the  strongest  trend  as  in  previous  studies.  Tangent  ellipses  and  the  upper  and 
lower  halves  of  this  form  are  quite  select! /e  for  this  meaning. 


Discussion 

The  reduced  test  battery  resulted  in  much  more  significant  trends  toward 
meaning.  Symbols  with  strong  selectivity  and  strong  preference  may  give  an 
advantage  in  rapid  location  on  a  mixed  display. 

It  is  also  evident  that  within  this  test  battery  the  basic  trends  of  our  earlier 
studies  (22)  continue  to  hold  and  tend  to  be  confirmed  by  their  repetition.  They  also 
agree  Dasically  with  the  ’  guarded"  conclusions  of  Torre  and  Sanders  (56): 

1.  Simple  enclosed  forms  tend  toward  a  friendly  meaning.  The  circle  ranks 
highest  followed  by  the  ellipse,  square,  and  horizontal  rectangle  in  that  order. 

2.  Simple  enclosed  forms  which  are  not  highly  selective  in  meaning  include 
the  diamond,  triangle,  and  vertically  oriented  rectangle.  These  symbols  also  had  a 
low  total  use  in  the  free -drawing  test  and  may  be  considered  as  having  low  salience. 

3.  Open  angular  straight-line  forms  are  strongly  associated  with  the  hostile 
meaning  and  are  apparently  highly  salient.  Open  forms  which  approximate  a  triangle 
are  superior  to  complete  triangles  both  in  saliency  and  selectivity, 

4.  Complex  curved -line  forms,  both  open  and  closed,  lean  toward  the  unknown 
meaning. 
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EXPERIMENT  IV  -  CONFUSION  STUDIES  -  SECOND  SERIES 


The  results  of  Series  I  findings  suggested  that  the  triangle  and  diamond  are 
very  poor  symbols  and  the  square  is  only  slightly  better.  The  next  logical  step 
should  be  locating  the  causes  of  increased  time  and  errors  with  these  symbols. 

It  seemed  possible  that  the  major  problem  was  caused  by  the  use  of  these 
three  similar  forms  in  a  single  set  rather  than  by  inadequacies  of  the  individual 
symbols.  On  this  theory,  sets  that  do  not  repeat  the  three  similar  forms  should 
clarify  the  findings. 

It  seemed  logical  to  enlarge  the  current  TSQ51  code,  which  includes  a  circle, 
90°  diamond  and  a  "U"  formed  from  the  lower  75  percent  of  an  octagon.  These 
forms  represented  ASTs  in  the  friendly,  hosilte  and  unknown  categories  respectively. 
Choosing  symbols  for  friendly  and  hostile  TBMs  was  necessary  to  complete  the  code. 

Some  familial  relationship  within  the  friendly  and  hostile  meanings  was 
considered  advisable  for  rapid  interpretation  of  the  total  air -defense  picture. 
Available  forms  were  studied  with  this  grouping  in  mind.  Findings  in  Experiment  III, 
the  study  of  stereotyped  meaning,  were  also  considered. 

Of  all  23  basic  symbols,  the  oval  seemed  an  ideal  choice  for  a  friendly  TBM. 
The  large  half -circle  had  also  proved  significantly  non -hostile  and  had  been  found 
to  give  excellent  performance  in  conjunction  with  the  circle  during  Experiment  n. 

The  oval  and  large  half -circle  were  selected  to  represent  a  friendly  TBM. 

The  hostile  TBM  required  a  straight-line  angular  form  to  pair  with  the  90° 
diamond.  Selection  was  further  limited  by  the  presence  of  the  open  octagon.  After 
reviewing  results  in  Experiment  III,  the  "vee"  or  pointer  was  selected  because  of  its 
strongly  hostile  meaning.  Although  this  form  approximated  the  triangle,  it  had 
proved  signifies!  :ly  stronger  and  more  hostile  in  the  meaning  experiment.  A  vee 
with  a  45°ang1'  was  selected  as  differing  most  radically  from  the  diamond.  In 
conjunction  with  the  half -circle,  an  apex -up  orientation  offered  the  best  possibility. 

If  the  oval  was  used  with  the  major  axis  vertical,  the  pointer  would  be  oriented  with 
the  apex  to  the  right.  Both  orientations  presented  the  open  area  in  an  orientation 
differing  from  the  "U". 

There  was  also  the  possibility  that  providing  a  redundant  cue  such  as  a  size 
difference  could  Increase  the  discernibility  of  the  circle  and  diamond  cue.  This 
was  considered  in  the  test  design. 
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Series  II  Design 

The  Series  II  test  design  again  presented  four  sets  of  symbols  and  followed 
the  location  design  of  Series  I.  Internal  linear  modifiers  v  .re  included  for  clutter 
effect.  The  complete  sets  .are  illustrated  in  Figures  7  through  10. 

The  circle,  90°  diamond  and  the  partial  octagon  dimensioned  to  1/4 -inch  width 
and  height  were  common  to  sets  a  and  b.  For  the  remaining  symbols,  set  a  used  the 
enclosed  upper  half  of  a  3/8 -inch  diameter  circle  and  a  1/4-inch  high  45°  pointer 
with  apical  angle  at  the  top.  In  set  b  these  symbols  were  replaced  by  a  35°  ellipse 
with  a  5/16-inch  vertical  major  axis  and  the  45°  pointer  with  apical  angle  to  the  right. 

Sets  c  and  d  followed  the  symbol  forms  of  set  a  with  size  differences.  In  c 
the  circle  diameter  was  reduced  to  3/16  inch.  For  d  the  reduced  circle  size  was 
coupled  with  a  1.4x  increase  in  the  90°  diamond  to  make  the  size  difference  evident. 

With  this  design,  comparison  of  sets  a  and  b  should  reveal  the  relative  merits 
of  che  half  circle  and  ellipse  as  well  as  of  the  90°  rotation  of  the  pointer.  Sets  a,  c 
and  d,  with  increasing  size  ratio  between  the  diamond  and  circle,  should  identify  the 
advantages  of  size  as  a  redundant  cue  in  patterning  symbols. 

Subjects  and  Procedure 

Subjects  from  Experiment  I  through  in  again  participated  in  this  series.  Test 
procedures  and  fatigue  ordering  were  identical  to  Series  I.  Since  the  20  subjects 
had  previous  experience  on  tests  of  this  type,  a  learning  advantage  could  have  been 
maintained  over  the  three -week  period  between  testing  sessions. 

To  obtain  some  measure  of  contamination  of  results  rom  the  effects  of  previous 
testing,  two  symbols  from  Series  Id  --  the  square  and  triangle  --  were  selected  for 
retest  during  the  new  series.  To  preserve  a  fatigue  order,  subjects  in  groups  1  and 
4  were  retested  on  the  triangle  before  the  Series  II  sequences  and  on  the  square  after 
all  tests  of  Series  II  were  completed.  Subjects  in  groups  3  and  4  were  given  the 
retest  material  in  reverse  order. 


43 


Results 


Table  9  gives  total  times  and  errors  on  the  elements  of  this  test. 

It  is  immediately  evident  that  times  have  improved.  Since  subjects  had 
previously  been  tested  in  Series  I,  experience  might  be  suspected  as  a  factor. 

Because  we  were  using  the  same  subjects,  we  had  guarded  against  this  factor 
by  including  pre-  and  post -tests  on  two  forms  from  Series  I.  There  were  no  signifi¬ 
cant  changes  in  either  time  or  errors,  nor  did  the  pre-  and  post -tests  reveal  signifi¬ 
cant  differences.  All  differences  appeared  to  derive  from  the  chosen  symbol  sets. 

Times  for  the  square  in  the  original  series  give  a  mean  of  1.047  minutes  and 
S2  of  .0485.  On  the  retest  the  mean  time  is  .986  and  S2  =  .0590.  Test-retest 
correlation  is  +.714.  From  the  correlated  data  formula  (44),  the  standard  error 
of  the  difference  is  calculated  as  .0399.  A  two  -tail  student's  test  of  differences 
between  means  at  the  .05  level  requires  a  t  =  D  =  2.093.  Thus,  with  the 

% 

obtained  S^,  the  difference  between  means  would  need  to  be  .084  or  greater.  Since 
the  obtained  difference  is  only  .061,  we  cannot  reject  the  hypothesis  that  no  change 
in  performance  occurred.  There  were  12  errors  in  the  original  test  and  only  8  on 
the  retest.  Hypothesizing  equality,  the  X1  value  is  less  than  1.0.  Since  a  X2  of 
3.84  is  required  for  P  =  .05,  df  =  1,  equality  cannot  be  rejected  and  no  ' 
significant  changes  are  evident  on  retesting. 

Times  for  triangle  in  the  original  series  average  1.1685  min,  S2  =  .05851. 

On  the  retest  the  mean  time  is  1. 1600  minutes  and  the  value  for  S2  =  .0501.  Test- 
retest  correlation  here  is  +.580,  considerably  lower  than  for  the  square  but  still 
a  correlation  significant  at  the  .01  level.  The  standard  error  of  the  difference 
between  means  is  calculated  as  .048.  A  standard  error  of  this  magnitude  requires 
a  difference  between  means  of  0. 1004  for  five  percent  level  of  significance  using  "t" 
distribution  as  above.  The  obtained  difference  is  .0085  minutes,  so  we  cannot  reject 
the  hypothesis  of  no  change  in  performance.  The  relatively  low  correlation  of  the 
test -retest  in  the  triangle  may  have  further  implications  for  learning  retention. 

There  were  22  errors  on  the  original  test  and  21  on  the  retest.  In  this  instance 
X2  =  0  and  equality  again  cannot  be  rejected.  This  indicates  that  Series  II  perfor¬ 
mances  which  are  improvements  over  Series  I  may  represent  an  improved  coding 
method. 

Factor  analysis  and  Newman -Keul  procedures  indicate  that  within  Series  n 
total  time  differences  exist  only  between  sets  b  and  d.  Set  d  times  are  significantly 
(.05  level'  lower.  Chi  aqua  re  statistics  gives  .01  significance  to  error  differences 
with  set  d  iow  and  set  b  high  on  errors.  Based  on  total  performance,  set  d  would  be 
chosen  over  all  other  combinations  in  Series  I  or  n. 
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To  demonstrate  the  effects  of  changes  within  the  sets  of  Series  II,  individual 
symbol  classes  were  analyzed  separately  with  the  set  design  considered  as  the 
treatment  variable.  Analysis  of  variance  and  the  Newman -Keul  procedure  were  run 
on  the  five  symbol -meaning  categories. 

TABLE  9 


Summary  of  Times  and  Errors  on  Experiment  IV,  Series  II 

N  =  20 


Pointer 

Circle 

Diamond 

U 

Half  Circle  Ellipse 

Total 

Set 

a. 

Size  constant 

T 

15.58 

15.40 

20.22 

17.03 

17.07 

85.30 

with  half  circle 

E 

3 

4 

16 

19 

10 

52 

b. 

Size  conr'ant 

T 

14.58 

18.13 

19.90 

14.22 

20.87 

87.70 

with  ellipse 

E 

10 

10 

17 

6 

48 

91 

c. 

Circle  smaller 

T 

15.96 

15.66 

20.59 

16.92 

16.45 

85.58 

E 

4 

0 

21 

30 

5 

60 

d. 

Circle  smaller 

T 

15.60 

12.99 

19.57 

17.02 

16.97 

82.15 

and  diamond 
larger 

E 

9 

1 

15 

24 

11 

60 

T  =  Sum  of  time  in  minutes  for  20  subjects  on  a  run  or  sequence 
E  =  Errors  of  omission 

Sets  are  illustrated  in  Figures  7  through  10. 

Individual  symbols  follow  an  interesting  trend.  Sets  a,  c,  and  d,  which  used 
the  identical  form  sets  with  the  relative  size  varied,  evidence  the  best  time  on  the 
circle  and  pointer,  slightly  slower  times  on  the  half  circle  and  the  U,  and  the  poorest 
time  (significant  at  .01  level)  on  the  diamond.  The  bulk  of  the  errors  are  on  the 
diamond  and  the  U. 

Set  b,  with  the  half  circle  replaced  by  a  vertical  ellipse  and  the  pointer  oriented 
horizontally,  demonstrates  an  entirely  different  trend.  The  pointer  and  the  U  now 
give  the  best  times  and  are  superior  to  the  other  three  forms  at  the  .01  level.  The 
circle  and  diamond  are  now  on  a  par  and  the  ellipse  proves  the  slowest  in  processing. 
Errors  are  also  significantly  higher  in  this  set.  The  ellipse  is  the  greatest  offender 
and  the  circle  also  shows  increased  errors. 
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The  effect  of  total  set  context  on  ;**j  Individual  symbol  is  not  significant  for  the 
pointer  or  diamond.  In  the  case  of  the  u,  set  b  is  superior  at  the  .01  level.  For 
the  circle,  set  b  showed  the  poorest  time  (.01  level).  These  findings,  added  to  the 
fact  that  the  ellipse  is  similarly  inferior  (.01  level)  to  the  half  circle  present  in  the 
other  sets,  suggest  the  ellipse  as  the  cause  of  both  changes.  TTiis  is  noted  for 
fuither  study. 

The  circle  time  is  significantly  better  (.01  level)  in  set  d.  In  this  instance 
reducing  diameter  of  the  circle  and  increasing  the  size  of  the  diamond  may  have 
optimized  the  patterning  effects.  There  is  also  a  slight  but  not  significant  reduction 
of  time  on  the  diamond  in  this  set. 


Discussion 

All  sets  of  Series  II  show  a  time  advantage  over  all  sets  of  Series  I.  Experience 
is  not  the  explanation,  since  items  from  the  previous  series  used  as  pre-  and  post- 
tests  show  no  improvement. 

It  would  at  first  seem  that  removing  the  triangle  created  all  changes,  but  the 
triangle  was  not  truly  replaced.  The  pointer  is  simply  a  triangle  minus  its  base. 

We  sharpened  the  angle  of  this  pointer  by  reducing  it  from  60  to  45°.  At  the  same 
time  we  increased  the  upper  angle  of  the  diamond,  magnifying  the  difference  of  the 
angular  portions.  Which  of  these  factors  is  responsible  for  the  very  large  time  and 
error  reductions  of  the  new  symbol?  The  angular  advantage  for  differentation  applied 
also  to  the  diamond.  Here  times  and  errors  are  reduced  only  slightly.  This  would 
seem  to  indicate  that  the  open  form  leads  to  the  improved  performance. 

On  the  other  hand,  there  Is  a  second  open  form,  the  U.  Performance  is  rather 
poor  on  this  symbol  except  in  set  b,  where  the  half -circle  was  replaced  with  an 
ellipse.  Since  this  change  improves  performance  on  the  U,  there  is  only  one  possible 
contributing  factor:  the  base  diameter  on  the  half -circle  must  be  patterning  with  the 
lower  line  of  the  U.  This  seems  quite  possible,  since  both  time  and  errors  on  the 
half -circle  have  increased  slightly  over  the  same  form  in  Series  I. 

The  less  desirable  changes  in  set  b  are  almost  entirely  related  to  time  and 
errors  on  the  circle  and  the  ellipse.  There  see  tittle  doubt  here  that  the  circle 
and  ellipse  are  competing  figures  simply  because  they  are  both  pure  curved-line 
forms. 

Considering  the  various  interference  effects,  it  seems  probable  that  the  best 
set  of  symbols  should  be  as  completely  different  as  possible.  This  brings  up  a  real 
problem.  We  do  not  seem  to  have  five  simple  and  entirely  different  symbols  in  our 
repertoire . 
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EXPERIMENT  V 


Results  of  the  previous  experiments  pointed  toward  the  need  for  a  set  of 
symbols  with  strong  differences  on  some  dimension.  It  appeared  that  the  circle 
could  be  quickly  patterned  against  a  ground  of  straight-line  forms  and  that  it  held 
its  advantages  even  in  the  presence  of  an  enclosed  half -circle  of  equal  height. 

When  an  ellipse  was  substituted  for  the  half -circle,  the  circle  required  more  time 
than  it  had  with  the  half -circle  and  the  ellipse  required  far  more  time  than  the 
half -circle  had. 

The  triangle  and  diamond  also  apparently  interacted.  When  a  pointer  replaced 
the  triangle,  it  was  an  excellent  symbol  whether  oriented  on  a  vertical  or  horizontal 
axis.  The  diamond  was  poor  in  conjunction  with  either  the  triangle  or  pointer  and 
thus  gave  evidence  of  being  a  poor  symbol.  One  problem  was  evident  throughout 
the  test  sessions  --  subjects  repeatedly  described  problems  with  triangles  and 
illustrated  by  pointing  to  diamonds.  Since  all  of  the  subjects  played  cards,  this  was 
an  unusual  situation. 


The  two  open  forms  of  Series  II  did  not  confuse.  There  was  some  evidence 
that  the  open  octagon  had  interference  from  the  enclosed  half -circle  since  this  form 
improved  in  both  time  and  errors  when  the  ellipse  replaced  the  half -circle.  In  this 
case,  the  similarity  of  the  two  interfering  forms  existed  only  in  the  presence  of  a 
horizontal  straight  line. 

To  further  identify  dissimilarities  that  made  a  set  of  geometric  shapes 
distinctive,  we  surveyed  written  comments  of  the  subjects.  In  the  questionnaire 
at  the  end  of  Series  I  of  Experiment  2,  all  subjects  had  stated  that  circles  and  half- 
circles  were  easiest.  Eighteen  also  complained  of  the  triangle  and  diamond  as  confusing. 

Only  one  subject  said  he  had  detected  a  patterning  of  the  test  material.  He 
described  three  of  the  10  symbol  patterns  accurately  enough  to  prove  that  he  had 
indeed  recognized  them.  He  said  that  circles,  which  were  his  first  assigned 
targets,  seemed  to  pop  up  and  appear  as  a  unified  design  upon  a  mixed  field.  He 
began  to  look  for  a  similar  effect  when  searching  for  other  targets.  By  changing 
his  search  method,  he  was  able  to  locate  the  half -circles  as  a  total  pattern  but 
other  symbols  were  seen  individually.  Since  the  patterns  were  identical  for  all 
symbols,  apparently  the  "pop  up"  was  less  dramatic.  This  subject  had  diamonds 
for  his  final  target.  In  spite  of  his  pattern  discovery  earlier  in  the  test,  he  took 
50  percent  more  time  for  the  diamond  search  than  he  had  for  the  circle. 

After  Series  II,  this  same  subject  said  the  pointer  was  good.  He  modified 
his  search  as  a  diagonal  movement  across  the  target  material  and  the  pointers 
popped  up  even  better  than  the  circles;  however,  he  insisted  the  patterns  were 
different  for  the  two  targets.  On  retest  with  the  triangles  from  Series  Id,  he 
could  not  locate  a  pattern  nor  could  he  describe  a  better  search  method. 
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Twelve  other  subjects  commented  on  their  ability  to  locate  the  pointer  by  a 
diagonal  search.  All  stated  that  a  random  search  seemed  easiest  for  the  circle 
and  half -circle.  Only  two  of  the  subjects  failed  to  mention  advantages  of  different 
search  methods. 

Series  lid  also  Indicated  some  advantages  in  providing  redundant  size  cues. 
Location  times  for  the  circle  were  most  rapid  when  it  was  appreciably  smaller 
than  all  other  symbols. 

The  combination  of  results  from  testing  and  interviews  suggested  that  three 
lines  of  thought  be  pursued  in  attempting  a  better  series.  The  symbols  should  be 
as  unlike  as  possible,  they  should  include  redundant  cues,  and  they  should  permit 
a  radar  operator  to  sec  them  as  a  pattern  if  he  adopted  an  appropriate  search  method. 

We  now  had  three  good  symbols  for  a  series.  The  circle  and  half -circle 
paired  well  for  a  friendly  meaning.  The  pointer  was  an  excellent  hostile  symbol 
and  could  possibly  be  paired  with  a  form  which  would  give  better  results  than  the 
diamond.  Since  none  of  these  three  forms  had  a  vertical  component,  a  square  or 
plus  sign  offered  a  good  choice  although  both  forms  had  been  related  to  a  friendly 
meaning  in  previous  experiments.  The  "unknown”  symbol  remained  a  problem. 
Intersecting  arcs  oriented  vertically  were  chosen  as  most  unlike  the  remainder  of 
the  series.  This  form  had  an  apparent  major  axis  in  the  vertical  dimension  and 
thus  differed  from  the  half -circle  and  horizontal  pointer.  It  presented  both  horizontal 
and  vertical  open  areas  as  opposed  to  plus  sign  with  its  open  areas  on  the  obliques. 

This  symbol  had  also  been  strongly  associated  with  the  '  unknown''  meaning  in 
experiment  II. 

The  circle  was  further  reduced  in  size  to  5/32 -inch  diameter,  with  the  half- 
circle  held  at  the  3/8 -inch  diameter.  The  pointer  continued  at  1/4"  major  dimension 
and  was  oriented  with  apical  angle  to  the  right.  The  intersecting  arcs  were  based  on 
a  3/16-inch  diameter  circle  to  fall  between  the  sizes  of  the  other  curvilinear  forms. 
Looking  back  at  this  design,  it  seems  probable  that  a  slightly  larger  dimension  might 
have  been  desirable . 

These  four  symbols  were  common  in  two  designs  of  Series  III.  The  fifth 
symbol  was  a  1/4-inch  diagonal  square  in  set  Ilia  (Fig.  11)  and  a  plus  sign  with 
I/4-inch  legs  in  set  DIb  (Fig.  12).  Alphabetic  modifiers  were  included  for  the 
clutter  -eFect. 

The  third  set  of  this  series  was  a  second  contractor  design  based  on  a  logical 
"lexicon."  In  this  instance,  all  symbols  were  1/4 -inch  high  and  were  made  up  of 
straight-line  segments.  Pairing  within  the  basic  friendly  or  hostile  meaning  was 
extremely  strong  with  circle  and  ellipse  as  friendly  AST  and  TBM  against  an  X  and 
a  plus  sign  for  the  two  hostiles.  The  "unknown"  was  an  isosceles  triangle  with  the 
45°  apical  tingle  down.  These  were  the  nouns  of  the  series.  The  modifiers  or 
adjectives  recommended  as  part  of  the  lexicon  were  used  for  clutter  in  the  presentation. 
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This  set  has  been  designated  as  Series  IIIc  and  is  illustrated  in  Figure  13.  The 
segmented  ellipse  with  its  vertical  major  axis  has  a  definitely  pointed  appearance 
as  in  the  design  recommended  by  the  contractor.  While  this  set  violated  all  three 
tentative  design  rules  derived  from  our  earlier  st„d  's,  it  had  the  advantage  of 
presenting  a  logically  derived  language. 


Subjects  and  Procedure 

At  this  point  there  were  10  sets  of  modified  symbols.  The  unmodified  la  set 
was  omitted.  Ten  new  subjects  matching  our  original  group  in  age,  education  and 
technical  background  were  selected  for  a  comparative  study.  All  subjects  scored 
9  or  higher  on  the  near -point  tests  of  the  Bausch  &  Lomb  Ortho -Rater.  No  attempt 
was  made  to  match  subjects  oi.  ability.  While  familiarization  runs  were  administered 
with  symbols  not  included  in  any  of  the  test  series,  there  was  no  attempt  at  training 
to  a  plateau. 

The  sets  were  numbered  from  one  to  10  and  testing  orders  were  determined 
usinv  a  random -number  desien.  following  a  10  x  10  matrix  with  no  set  permitted 
in  the  same  row  or  column  a  second  time.  In  accordance  with  this  design,  each 
set  was  administered  one  time  in  each  of  the  10  sequence  positions.  Testing  orders 
were  assigned  randomly  to  the  10  subjects-  Individual  target  presentations 
followed  ascending  and  descending  five -item  orderly  rotations.  These  procedures 
resulted  in  a  reasonable  balancing  of  fatigue  and  learning  effects  on  a  group  of 
unmatched  subjects. 

Testing  and  scoring  was  identical  to  previous  experiments.  Rest  periods 
followed  alternate  sets,  with  subjects  cleaning  the  test  books  during  this  time. 


Results 

Errors  and  times  for  the  10-subject  pool  are  summarized  in  Table  10.  Total 
scores  for  a  five -symbol  set  show  progressive  improvement  with  each  design  change 
growing  out  of  a  previous  experiment.  Sets  in  Series  I  and  II  follow  the  trends  of 
the  earlier  experiments.  The  logical  lexicon,  Series  IIIc,  proves  the  least  satis¬ 
factory;  Series  Ilia  gives  the  best  performance.  The  time  ratio  of  the  best  to  worst 
set  is  1:  1.6  and  error  ratio  is  1:6. 

Variances  follow  a  similar  trend,  with  the  greatest  2.285  for  Hie  and  the 
smallest  .637  for  Ilia.  The  resulting  Fmax  of  3.59  fails  to  reject  homogeneity 
of  variance.  The  Newman -Keuls  test  separates  times  into  three  classes.  The  HE L 
designs  Ilia  and  Illb  are  superior  to  the  TSQ51  sets  of  Series  II  a c  the  .05  level. 

All  sets  of  Series  I  and  the  lexicon,  IIIc,  are  inferior  at  the  .01  level. 
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Summary  of  Performance  on  Experiment  V 


=  Symbol  and  modification  used 
=  Errors  for  10  subjects 
=  Total  time  in  minutes  for  10  sm,,ects 


Grouping  errors  into  the  mean  time  classes  indicated  by  the  Newman -Keuls 
procedure  and  weighting  expected  errors  by  the  number  of  sets  in  each  class,  gives 
an  X2  of  49.28,  rejecting  equality  at  the  .001  level. 

The  10  forms  under  each  recommended  meaning  category  are  next  compared 
for  effectiveness  within  their  code  complex  following  a  simple  analysis  of  variance 
design  (n=10,  k=10).  Homogeneity  of  variance  is  tested  by  the  Fmax  statistic 
(Fmax  .95(10,  9)  =  9.91).  Findings  on  this  check  are  mentioned  only  when 
homogeneity  of  variance  is  rejected.  Treatment  comparison  uses  the  Newman -Keuls 
procedure. 


F riendly  AST 

The  friendly  AST  was  represented  by  a  circle  in  all  10  sets.  Total  symbol 
location  times  for  the  10  subjects  varies  from  6.11  minutes  in  lid  to  9.00  minutes 
for  lb.  The  Newman-Keuls  procedure  reveals  significant  differences,  clumping 
symbols  into  three  classes.  The  first  class  contains  only  lid  which  is  superior  to 
symbols  in  the  second  class  at  the  .05  level.  The  third  class,  containing  lib,  IIIc, 
lb,  and  Ic,  is  inferior  to  all  others  at  the  .01  level. 

The  best  performance  on  this  target  occurred  where  the  circle  had  the 
smallest  relative  size  in  the  code  set.  Since  the  circle  had  been  uecreased  in 
size,  this  advantage  cannot  be  attributed  to  pure  discernibility. 

The  poorest  performance  on  the  circle  occurred  in  the  presence  of  a  square 
to  the  same  size  dimension  or  an  oval.  Both  of  these  designs  had  been  considered 
faulty  in  our  earlier  experiments.  The  interference  effect  between  the  oval  and 
ellipse  is  strong  enough  to  leave  no  question.  The  effect  of  the  square  on  the  circle 
is  less  definite.  The  square  in  Ilia  had  been  made  larger  than  the  circle  to  counter¬ 
act  the  suspected  interference  and  times  on  the  circle  had  improved  as  hypothesized. 
This  fact  might  bear  more  weight  if  the  linear  modified  circle  and  square  of  Series  I 
had  not  proved  superior  to  the  same  symbols  with  internal  or  external  alphabetic 
modifiers.  Casting  about  for  an  explanation  of  this  difference  we  studied  the  test 
sheets  for  the  three  sets.  The  circles  appeared  much  smaller  on  Ic.  This  was  not 
an  artifact.  We  had  made  an  error  in  our  drawings  and  the  circle  was  smaller, 
providing  a  redundant  cue. 


Friendly  TBM 

The  friendly  TBM  is  an  ellipse  in  lib  and  IIIc  and  a  half -circle  in  all  other  sets. 
Times  vary  from  5.57  for  Hid  to  10.71  for  IHc.  The  Newman-Keuls  test  again  gives 
three  classes  of  difficulty,  with  the  first  class  superior  to  the  second  at  the  .05  level 
and  the  third  class  inferior  to  all  others  at  .01. 


Interference  between  the  circle  and  ellipse  is  again  apparent.  The  class  3, 
or  poorest  targets,  are  in  lib  and  IIIc,  the  two  sets  with  this  combination  of  symbols. 
These  two  sets  also  account  for  25  of  the  39  errors  within  this  meaning.  In  lib  there 
is  also  a  suggestion  of  interference  between  the  diamond  and  ellipse. 

Class  1  symbols,  or  the  best  targets,  are  within  sets  Ilia  and  IIDj.  Apparently 
neither  the  horizontal  component  r'f  the  square  nor  the  plus  sign  interferes  with  the 
base  line  of  the  half -circle. 

The  remaining,  class  2,  sets  combine  the  half -circle  with  either  the  open 
octagon  or  the  triangle.  Of  these,  the  ooen  octagon  results  in  greater  changes  of 
times  and  errors  for  the  half -circle. 


Hostile  AST 

The  times  for  hostile  AST  vary  from  7.57  for  the  square  of  Ilia  to  12.56  for 
the  diamond  of  lb.  The  Newman-Keuls  test  again  divides  into  three  classes. 

Tne  best,  eiass  1,  symbols  are  the  square  and  the  plus  sign  within  Ilia  and 
IHb.  These  are  superior  at  the  .05  level  to  the  90°  diamonds  of  the  four  sets  of 
Series  II.  The  third  class  targets  represented  by  Scries  I  and  IIIc  are  inferior  to 
all  others  at  the  .01  level.  The  vertically  elongated  diamonds  of  the  three  sets  of 
Series  I  also  account  for  53  of  the  89  errors,  suggesting  interference  by  the 
triangle  included  in  this  code. 

It  is  interesting  that  the  square  and  plus  sign  which  rank  best  in  the  hostile  AST 
meaning  and  within  their  code  complexes  are  poor  symbols  in  other  codes. 


Hostile  TBM 

Hostile  TBM  targets  range  from  5.89  for  Series  Ilia  to  12.77  for  Series  IIIc. 
FmaX  8.70  does  not  reject  homogeneity  of  variance  but  does  approach  significance. 
This  high  variance  ratio  is  due  to  the  "lexicon"  or  Series  IIIc  which  has  three  times 
the  variance  of  any  other  set. 

The  Newman-Keuls  test  separates  hostile  TBM  into  two  classes.  All  tests 
using  the  pointer  are  superior  at  the  .01  level  to  the  triangle  of  Series  I  tests  and 
the  plus  sign  of  Series  Me.  Errors  follow  a  similar  trend. 

If  the  lexicon  set  is  removed  from  die  analysis,  FmaX  drops  to  3. 11  and  fails 
to  reject  homogeneity  of  variance  F  (9,  9)  =  9.45  .  The  two  class  divisions 

remain,  with  the  pointers  superior  to  triangles  at  the  .01  level.  When  the  cases 
are  reduced  to  ihe  six  sets  using  the  pointer,  set  Ilia  is  superior  to  all  others  at  the 
5  percent  level.  This  is  the  only  set  which  has  no  other  open  straight-line  form 
within  its  code. 
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The  "unknown"  targets  were  represented  by  a  square  In  Series  I,  an  open 
octagon  in  Series  II,  intersecting  arcs  in  Series  Ilia  and  b,  and  a  closed  triangle 
(exactly  equivalent  to  the  pointer  with  apex  down),  in  set  IILc.  Fm  of  7.24  does 
nnt  reach  a  significant  level.  The  variances  ere  quite  irregular  with  differences 
unrelated  to  symbol  form. 

The  Newman-Keuls  comparison  of  means  divides  this  group  into  only  two 
classes  with  some  overlap.  The  entire  pattern  suggests  that  the  data  were  not 
suitable  for  this  procedure.  "Unknown"  targets  are  next  considered  within  their 
shape  categories. 

The  intersecting  arcs  of  Ilia  and  Illb  have  mean  scores  of  .760  and  .886 
minutes  respectively.  Pearson  product  moment  r  between  the  two  tests  is  +.916. 

The  standard  error  of  the  difference  between  the  two  means,  correlated  data,  (44) 
is  .047  giving  a  t -score  of  2.68.  These  differences  are  significant  between  .01  and 
.025  with  the  advantage  for  set  IUa.  Since  the  sets  differ  only  in  the  hostile  AST 
target,  any  differences  must  be  associated  with  this  symbol.  In  Ulb  the  plus  sign 
presents  four  symmetrical  open  areas.  While  these  areas  are  rotated  45°  from  the 
open  sections  of  the  intersecting  arcs,  there  is  apparently  some  interference  within 
this  five -symbol  structure.  The  square  of  Ilia  is  more  desirable  for  rapid  recogni¬ 
tion  of  the  intersecting  arcs. 

Sets  of  Series  II  used  an  open  octagon  for  the  "unknown”  symbol.  Size  varia¬ 
tions  between  the  circle  and  diamond  were  present  in  the  series,  with  the  circle 
relatively  smaller  progressing  from  a  to  c  to  d.  Set  b  held  to  the  size  of  a,  but. 
substituted  an  oval  for  the  half -circle  used  in  other  sets  of  the  series.  Analysis  of 
variance  of  the  unknown  symbols  for  these  four  sets  gives  an  -F  of  2.53. 
(Fmax95  (4,  9)  =  6.31).  Total  times  vary  from  6.74  for  set  b  ton^5^6  for  set  d. 

The  Newman-Keuls  test  divides  into  two  classes  with  set  lib  superior  to  others  at 
the  .05  level.  This  set  also  shows  the  fewest  errors  on  the  open  octagon.  The 
interference  of  the  diameter  of  the  half -circle  is  again  suspect  as  it  was  in 
Experiment  III. 

All  sets  of  Series  I  used  a  square  for  the  unknown  target.  Total  times  vary 
from  9.92  for  set  d  to  11.96  for  set  b.  Fmax  =  2.40  and  does  not  reject  homogeneity 
of  variance  Fmax95  (3,  9)  =  5.34  .  The  Newman-Keuls  test  shows  set  d  superior 
to  b  and  c  at  the  .05  level.  This  difference  is  attributed  to  the  accidental  reduction 
in  circle  size  described  above. 


Discussion  of  Individual  Symbol  Study 

Circle 

The  circle  has  been  discussed  totally  as  a  friendly  AST.  Results  shov  the 
circle  is  best  when  relatively  smaller  than  other  symbol;:..  An  ellipse  withir  the 
code  degrades  performance  on  the  circle.  The  presence  o*  a  square  results  in 
some  degradation,  but  it  is  minimized  if  the  square  is  lary  than  the  circle. 

There  is  some  evidence  that  the  intersecting  arcs  of  a  90°  di  .mond  of  the  same 
size  also  cause  an  interference  effect. 

Half -Circle 

The  upper  half -circle  with  enclosing  diameter  proved  to  be  an  excellent  symbol. 
Performance  on  this  target  was  degraded  by  presence  of  the  open  octagon  and  possibly 
by  the  triangle.  Maximum  efficiency  with  this  target  was  obtained  in  combination  of 
the  circle  with  half -circle  diameter  at  least  twice  the  diameter  of  circle,  pointer  with 
apex  horizontal,  intersecting  arcs  and  either  the  square  or  plus  sign.  Within  this 
complex  the  diagonal  of  the  square  should  be  related  to  the  diameter  of  the  circle 
in  an  8:5  ratio. 

Ellipse 

The  ellipse  was  used  in  only  two  sets.  In  both  instances  performance  was 
inadequate  with  elongated  times  and  increased  errors.  The  circle  in  both  sets  was 
probably  the  major  contributing  factor,  and  there  is  some  evidence  of  interference 
with  the  diamond.  Since  the  circle  appears  to  be  an  ideal  target  in  terms  of  time, 
errors  and  generation  by  current  equipment,  the  ellipse  was  not  studied  further. 

Diamond 

The  diamond  proved  a  generally  poor  target  in  terms  of  times  and  errors.  It 
is  very  inferior  when  combined  with  a  triangle  of  nearly  equal  apical  angle.  When 
a  pointer,  or  open  triangle  of  strongly  different  apical  angle  is  present,  performance 
is  still  far  from  ideal. 

Pointer 

The  pointer,  or  open  triangle,  with  about  45°  angle  is  an  ideal  target  in  either 
horizontal  or  vertical  rotation.  It  can  be  used  with  the  circle,  half -circle,  90° 
diamond,  square,  plus  sign  or  intersecting  arcs  with  little  degradation.  This 
symbol,  like  the  circle  and  half -circle,  should  be  considered  in  any  code  set. 

The  six  pointers  used  in  the  various  sets  have  been  subjected  to  analysis  of 
variance.  Fmax  of  2.44  is  not  significant  (Fmax  .95(6,  9)  =  7.80.  The  times 
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ranged  from  5.89  for  Ilia  to  7.66  for  Ila.  Newman-Kculs  testing-  gives  Ilia  an 
advantage  over  the  remainder  at  .01  level  of  significance.  There  are  no  other 
significant  differences.  This  symbol  is  good  in  both  times  and  errors.  It  may  be 
used  with  a  circ  le,  half -circle,  90°  diamond,  open  octagon,  plus  sign  or  intersecting 
arcs.  It  may  also  be  used  with  an  ellipse  if  the  major  axes  of  the  two  symbols  are 
90°  apart.  The  ideal  combination  within  our  five -symbol  sets  is  Ilia,  which  includes 
no  other  open  straight-line  forms. 

Square 

The  square  was  present  in  all  of  Scries  I  and  in  Ilia.  Analysis  of  variance 
shows  an  Fmax  of  3.83.  (Fmax95  (3,  9)  =  5.35)  and  fails  to  reject  homogeneity 
of  variance.  Times  vary  from  7.57  for  Ilia  to  11.96  for  lb.  Errors  are  low  and 
about  equal.  The  Newman-Kculs  test  gives  Ilia  superiority  at  the  .01  level.  Set  Id 
is  superior  to  other  sets  of  Series  I  at  the  .05  level. 

In  the  two  poorest  sets,  the  diameter  of  the  circle  and  diagonal  of  the  square 
held  a  7:8  ratio.  In  Id  the  ratio  was  6:8  and  in  Ilia  5:8.  The  size  relative  to  the 
circle  seems  very  important  for  the  square. 

Half  Octagon 

This  symbol  is  discussed  under  "unknown"  targets.  Performance  was  best 
when  the  half -circle  with  enclosing  diameter  was  omitted.  Since  the  half -circle 
appears  to  be  an  ideal  target,  further  tests  were  omitted.  This  octagonal  form 
is  also  undesirable  because  of  degradation  on  the  CRT.  Its  use  is  not  recommended. 

Intersecting  Arcs 

The  intersecting  arcs  are  also  discussed  under  the  "unknown"  target  heading. 

It  is  possible  that  size  differences  would  further  improve  this  form  within  the 
recommended  codes.  Using  it  with  a  plus  sign  or  X  should  probably  be  avoided. 

This  symbol  has  minimal  effect  on  the  circle,  half -circle,  pointer  and  square.  It 
is  the  best  fifth  symbol  in  our  studies  to  date.  Further  experimentation  seems 
advisable  here. 

Triangle 

The  triangle  was  used  as  a  hostile  TBM  in  all  sets  of  Series  I  and  as  the 
"unknown”  target  in  IIIc.  Its  performance  was  poor  in  both  time  and  errors  in 
all  cases.  In  Series  I  the  presence  of  the  diamond  could  explain  poor  performance. 

In  IIIc  there  is  no  diamond  and  no  other  linear  enclosed  form,  but  the  pointed  design 
of  tne  segmented  oval  may  be  the  interference  factor.  In  this  set  the  apical  angle 
and  size  of  the  triangle  were  identical  to  the  pointer  which  tested  as  superior  in 
other  sets.  In  lid  the  pointer  is  paired  with  an  ellipse  with  no  evidence  of  interference. 
In  this  case,  the  target  was  a  true  ellipse  with  its  major  axis  vertical  to  contrast 


59 


with  the  pointer's  horizontal  major  axis.  Similar  design  changes  might  improve 
the  triangle  in  IUc. 

When  the  four  triangles  are  subjected  to  analysis,  Fmax  of  3.09  fails  to  reach 
significance  (Fqj-y  95(3.  9)  =  5.34).  The  test  times  vary  from  11.37  for  lid  to 
11.92  for  lib.  Dift  arences  are  not  significant  by  the  Newman -Keuls  test.  The 
triangle  gives  little  piomise  as  a  code  symbol  because  of  its  generally  poor  times 
and  large  error  components.  This  symbol  also  degrades  rapidly  to  a  blur  on  the 
CRT,  where  the  open  form  degrades  only  in  a  partial  filling  of  the  apical  angle. 
While  further  experimentation  on  the  triangle  might  be  pursued,  there  is  little 
encouraging  data  to  support  new  designs. 

Plus  Sign 

The  plus  sign  appeared  in  IUb  and  Ilic.  Total  times  were  8.03  for  Illb  and 
12.77  for  IUc.  The  Pearson  product  moment  r  is  .680  (for  95%  confidence,  N=10, 
r=.632).  The  index  of  forecasting  efficiency  is  .266.  Comparison  of  the  means 
using  the  correlated  data  formula  gives  a  t  score  of  5. 10  and  is  significant  at  less 
than  the  .01  level  (for  9  degrees  of  freedom  a  two -tail  t  test  requires  only  3.25  for 
.01  significance),  ihis  correlation  is  lower  than  generally  observed  in  our  data 
(.80  to  .93).  It  seems  probable  that  some  factor  other  than  the  subjects'  relative 
speed  is  influencing  time  performance.  The  large  number  of  errors  on  IUc  can  be 
attributed  to  the  presence  of  a  highly  similar  form  (the  X)  in  this  set.  This  inter¬ 
ference  factor  apparently  affected  subjects  unequally.  Our  data  would  suggest 
rejecting  the  use  of  these  two  symbols  in  the  same  code. 


Summary 

The  circle,  half -circle  and  45°  pointer  are  excellent  symbols  within  the  symbol 
complex  used  in  these  experiments.  Relative  sizes  are  important  and  should  conform 
to  the  design  of  Ilia.  The  pointer  may  be  appropriately  oriented  with  its  major  axis 
vertical  or  horizontal,  with  the  choice  related  to  other  symbols  within  the  code. 

The  square  is  probably  the  best  choice  for  a  fourth  symbol.  It  should  be 
relatively  larger  than  the  circle,  with  the  radius  of  circle  and  diagonal  of  square 
in  a  5:8  ratio.  The  square  wUl  probably  lose  some  effectiveness  if  combined  with  a 
diamond  or  triangle.  The  current  tests  did  not  include  the  square  in  combination 
with  a  plus  sign  or  X.  If  these  symbols  are  added  to  a  code,  they  will  require  testing. 

For  the  fifth  symbol  the  intersecting  arcs  hold  a  slight  advantage.  It  is  possible 
that  this  advantage  could  be  increased  with  size  changes.  The  symbol  loses  some 
of  its  effectiveness  in  combination  with  a  plus  sign  and  would  probably  be  quite 
unsatisfactory  with  an  X. 
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The  diamond  and  triangle  should  probably  never  be  used  regardless  of  angle 
difference  and  they  should  definitely  not  be  used  in  the  same  code.  These  symbols 
have  poor  discriminability  under  the  best  conditions  and  they  degrade  rapidly  on  a 
CRT. 

The  ellipse  should  also  be  avoided,  since  it  interacts  with  circles  to  slow 
processing  time  and  increase  errors  in  both  symbols.  The  ellipse  might  be  an 
excellent  symbol  in  a  code  without  the  circle  or  diamond,  but  the  advantages  of  the 
circle  are  great  enough  to  make  this  choice  of  dubious  value. 

The  plus  sign  and  X  should  probably  never  be  used  together;  however,  the 
plus  sign  is  satisfactory  with  the  45°  pointer.  The  X  wa3  not  tested  with  the  pointer. 

A  study  of  the  remaining  simple  forms  (Table  8)  which  may  be  easily  generated 
on  a  CRT  failed  to  suggest  other  symbols  which  might  be  expected  to  fit  well  Into  a 
discriminable  code.  All  have  features  of  strong  similarity  which  could  be  expected 
to  produce  problems.  Forms  from  Table  8  which  might  be  worthy  of  further  study 
are  the  double  ellipse  and  its  parts  (#9,  #10,  and  #13  of  the  table),  the  anchor  (#18), 
and  atom  (#21).  Any  of  these  symbols  might  be  used  for  the  "unknown”  target  with 
the  four  hostile  forms  of  either  nia  or  Illb  and  discriminability  could  probably  be 
maintained  by  enlarging  their  size. 

Our  studies  gave  no  evidence  concerning  advantages  or  disadvantages  of  either 
linear  or  alphanumeric  modifiers.  The  modifiers  were  considered  here  only  in  terms 
of  interaction  or  interference  with  basic  forms.  The  potential  advantages  of  modifier 
types  in  learning  and  information  content  should  be  investigated. 

While  the  logical  lexicon  is  the  poorest  in  terms  of  time  and  errors,  there 
may  be  advantages  in  its  grouping  into  hostile  and  friendly  meanings.  This  possibility 
is  investigated  in  Experiment  VI. 


EXPERIMENT  VI  -  GROUPING 


Our  experiments  had  been  based  on  the  theory  that  five  basic  symbols  should 
be  highly  aiscriminable  within  a  code  complex.  Using  time  required  and  errors 
made  in  locating  symbols  as  a  comparative  measure  within  and  between  sets,  we 
had  formulated  a  few  tentative  rules  for  designing  a  discriminable  code.  Applying 
these  rules  in  the  two  HEL  designs  reduced  time  and  errors.  The  best  HEL  design 
proved  highly  superior  to  the  logical  lexicon. 

While  the  lexicon  gave  the  poorest  test  performance,  this  set  had  been 
designed  on  the  theory  that  a  strong  familial  resemblance  was  desirable  within 
the  friendly  and  hostile  meanings.  Dr.  Hugh  Bowen,  who  designed  this  code,  felt 
that  the  basic  information  was  "friend,  "  "foe,  "  or  "unknown.”  He  stated  that  a 
hostile  plane  or  missile  presented  a  threat  and  that  the  type  of  threat  was  secondary 
information. 

HEL  codes  had  been  designed  on  the  theory  that  five  categories  needed  to  be 
differentiated.  We  had  considered  a  missile,  whethe  friendly  or  hostile,  as  the 
most  important  information  on  the  display.  Missiles  move  more  rapidly.  Decisions 
on  their  deployment,  threat,  vulnerability  and  destruction  must  be  made  quickly. 

The  fact  that  TBM  targets  appear  at  infrequent  intervals  and  small  densities  also 
pointed  to  the  need  for  using  the  most  salient  symbols  for  these  categories.  Familial 
forms  were  included  as  a  learning  and  retention  aid,  within  the  friendly  and  hostile 
meanings.  Familial  resemblance  was  descriptive  rather  than  perceptive,  since  all 
five  categories  were  important. 

The  difference  in  the  doctrine  behind  the  two  designs  represented  two  entirely 
different  conceptions  of  the  radar  operator's  task.  Since  we  really  know  very  little 
about  the  eventual  use  of  information  on  the  PPI,  it  seemed  advisable  to  run  additional 
tests  within  the  grouped -meaning  concept. 

The  codes  representing  the  two  extremes  on  performance  were  selected  for 
this  study.  The  best  HEL  design,  Ilia  represented  a  code  with  a  good  figure -ground 
reversal  on  all  symbols.  While  no  meanings  had  been  assigned  in  previous  testing, 
the  anticipated  use  included  a  weak  familial  resemblance.  The  circle  and  half -circle 
were  to  represent  the  friendly  AST  and  TBM  respectively.  Their  counterparts  within 
the  hostile  meaning  were  straight  line  forms,  the  square  and  pointer.  The  unknown 
target,  intersecting  arcs,  consisted  of  two  half -circles  and  might  be  considered  as 
fairly  closely  related  to  the  friendly  vmbols. 

The  lexicon,  IIIc  had  been  designed  with  closed  straight-line  approximations 
of  curved  forms  as  friend,  crossing  straight  lines  as  hostile  and  an  enclosed 
straight-line  form  for  unknown.  These  groupings  were  logical  and  could  be  easily 
explained.  Experiment  VI  was  designed  to  compare  the  two  information  doctrines 
using  the  symbol  sets  representing  the  extremes  of  our  code  batteries.  Since  learning 
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advantages  could  mask  performance  trends,  naive  subjects  were  selected  for  this 
study. 

Subjects 

Subjects  were  casual  troops  awaiting  assignment.  The  age  range  was  19  to  23. 
Since  none  of  these  men  had  ever  participated  in  an  experiment,  they  were  moderately 
apprehensive.  Their  education  and  technical  background  was  representative  of  the 
average  raw  recruit.  Skills  were  probably  below  the  level  required  for  radar  operators. 
While  acuity  was  not  measured,  all  subjects  had  passed  a  visual  acuity  test  on  entry 
physical  within  the  previous  two  months.  Distance  acuity'  scores  of  at  least  20/30 
are  thus  assured. 

Subjects  were  generally  interested  in  the  experiment  and  quite  cooperative. 

One  individual  was  eliminated  early  in  the  test  sequence  because  of  wandering 
attention.  Ten  subjects  participated  in  the  total  experiment. 

Procedure 

Subjects  were  given  a  short  description  of  the  radar  operator's  task.  Trial 
runs  on  two  randomly  selected  test  sheets  oriented  them  to  the  rest  material,  but 
there  was  no  attempt  to  bring  them  to  a  performance  plateau.  Instructions  were  to 
locate  and  mark  targets  as  rapidly  and  accurately  as  possible.  There  was  no 
additional  attempt  at  establishing  time  or  error  "sets. " 

Testing  covered  two  individual  tasks.  The  first  procedure  was  identical  to 
Experiment  V  and  required  location  of  individual  targets  in  the  two  sets.  A  rest 
period  followed  completion  of  these  test  runs.  In  the  second  procedure  subjects 
were  asked  to  locate  a  "family"  of  symbols.  This  was  the  only  experiment  with 
meanings  assigned  to  the  symbols . 

Subjects  were  numbered  consecutively  as  tested.  Fatigue  orders  presented 
Ilia  first  for  subjects  1  through  5  and  QIc  first  for  subjects  6  through  10.  The  two 
individual  symbol  runs  were  presented  prior  to  the  "familial"  runs  in  all  instances. 
Symbol  presentation  continued  in  orderly  rotation  from  subject  to  subject. 

Scoring  on  times  and  errors  followed  the  original  design. 
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Results 


These  subjects  were  slower,  less  accurate  and  more  variable  in  performance 
than  the  groups  who  had  been  trained  to  a  plateau.  Performance  trends  confirm 
earlier  findings  but  symbol  and  code  difference  are  less  exaggerated. 

Total  times  for  location  of  individual  symbols  are  43.18  minutes  for  Ilia  and 
57.22  for  IIIc  (contrasted  with  34.40  against  55.63  in  Experiment  V).  Subjects 
again  cend  to  hold  relative  speeds  fairly  constantly  during  the  test  session.  Total 
time  on  Ilia  versus  IIIc  correlate  at  r  =  +.912.  Differences  in  total  time  on  the 
two  series  remain  significant  (P<.01)  with  the  t  ratio  (correlated  data)  =  11.70. 

Series  Ilia  has  29  errors  against  79  in  IIIc  (contrasted  with  10  and  66  in 
Experiment  V).  Differences  give  an  X'  of  8.13,  representing  significance  at  P  <  .01. 

The  Newman-Keuls  procedure  divide  the  symbols  of  Ilia  into  two  groups  differing 
at  the  .05  level.  The  two  TBM  symbols,  pointer  and  half -circle  remain  superior  to 
the  circle,  square  and  intersecting  arcs.  The  total  time  increase  of  8.78  minutes 
(10  subjects,  5  runs)  over  the  earlier  group  is  almost  evenly  divided  between  the 
five  symbols.  While  errors  are  2.64  times  greater  than  found  in  E>pcriment  V, 
distribution  among  symbols  follow  the  trend  of  the  earlier  study. 

Performance  on  IIIc  matches  the  earlier  experiment  more  closely  with  1 .  S9 
minutes  total  increase  in  time  and  1.2  times  the  errors  of  experiment  V.  In  this 
instance,  changes  are  less  regular.  The  triangle  shows  improved  accuracy  coupled 
with  a  time  increase,  the  X  has  seven  times  the  errors  of  the  earlier  test  with 
times  also  elongated.  The  three  remaining  symbols  closely  approximate  results 
of  Experiment  V.  The  Newman-Keuls  procedure  separates  the  symbols  into  three 
classes  with  the  circle  and  ellipse  most  desirable,  the  X  and  plus  signs  poorest 
and  the  triangle  intermediate . 

All  findings  tend  to  lend  weight  to  Experiment  V  results.  Subjects  are  generally 
proportionately  rapid  or  slow,  accurate  or  inaccurate. 

Grouping  of  symbols  in  the  second  task  is  superior  on  IIIc.  The  circle  and 
eEipse  of  this  set  require  a  total  subject  time  of  11.34  minutes  with  only  two  errors 
while  the  circle  and  half -circle  of  Ilia  had  a  total  time  of  13. 13  minutes  with  six 
errors.  Correlation  approaches  zero  at  +.014.  Time  differences  are  significant 
at  the  .05  level. 

Hostile  symbols  follow  the  same  trend.  The  plus  sign  and  X  of  IIIc  require 
12.33  minutes  with  12  errors  as  compared  with  14.65  minutes  and  22  errors  on  Ilia. 
The  -.296  correlation  does  not  approach  significance.  Time  differences  are  again 
significant  at  the  .05  level. 
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Familial  likenesses  are  greater  in  Hie  and  these  symbols  do  result  in  good 
performance  when  friendlies  or  hostiles  are  grouped.  However,  the  entire  vest 
battery  must  also  include  an  unknown  target.  When  grouped  hostile,  grouped 
friendly  and  single  unknown  targets  are  combined,  111c  has  a  total  time  of  34.37 
minutes  and  22  errors  as  compared  with  36.99  minutes  and  22  errors  on  Ilia. 
Correlation  of  total  subject  times  approaches  zero  at  +.056.  While  the  lexicon 
series  shows  a  slight  time  advantage,  differences  in  times  and  errors  are  well 
below  the  .05  level  of  significance. 


Discussion 

The  set  designed  on  logical  and  familial  association  of  forms  is  unquestionably 
poor  when  the  five  categories  are  to  be  identified.  Only  the  Series  1  targets  are 
not  significantly  superior  in  time  or  errors. 

Sets  Ilia  and  Mb  designed  on  cues  from  earlier  test  series  are  consistently 
superior  with  times  and  errors  significantly  below  all  other  codes  tested. 

When  the  best  and  poorest  sets  were  repeated  with  a  naive  subject  group,  the 
lexicon  again  proved  less  satisfactory.  Differences  are  smaller  but  retain 
significance  at  the  .01  level. 

Tne  iexicon  had  been  designed  cn  the  theory  that  groupings  within  the  friendly- 
hostile  dimension  should  be  permissible.  For  this  reason  the  naive  subject  group 
was  retested  on  the  two  sets  with  their  task  oriented  toward  the  lexicon  doctrine. 

In  this  case  meanings  were  assigned  to  the  symbols  and  only  friendly,  hostile  and 
unknown  test  runs  were  made. 

The  lexicon  shows  dramatic  improvement  under  these  conditions.  Grouped 
hostile  and  friendly  symbols  are  now  superior  to  the  best  HEL  series.  When  the 
unknown  target,  which  completed  the  five  symbol  code,  is  considered  a  part  of  the 
total  performance,  the  two  sets  are  essentially  equal. 

If  we  are  correct  in  our  assumption  that  each  of  the  five  target  classes  must  be 
identified  rapidly  and  accurately,  the  two  HEL  designs  are  the  best  codes  tested  to 
date.  If  the  real  problem  is  simply  sorting  into  three  classes,  the  lexicon  is  equally 
satisfactory  and  future  studies  should  lean  toward  advantages  in  learning  the 
"objectives”  of  the  two  codes. 

It  is  difficult  to  justify  a  five -symbol  code  for  three  target  categories.  If  this 
doctrine  is  correct,  the  possible  advantages  of  reducing  to  three  symbols  must  be 
considered. 
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EXPERIMENT  VII  -  MEANING  WITHIN  CODE  CONTEXT 


We  had  found  that  the  discriminabiLity  of  a  symbol  was  strongly  affected  by 
the  presence  of  other  forms  in  the  same  code.  It  seemed  reasonable  to  suspect 
that  any  meaning  associated  with  a  form  might  also  vary  with  the  total  code  context. 
Since  test  materials  and  subjects  were  available,  we  designed  a  simple  intra-set 
meaning  experiment. 


Procedure 

Confusion  test  sheets  presented  the  various  forms  of  a  set  in  a  realistic  distri¬ 
bution  and  were  ideally  suited  for  a  study  of  symbol  meaning  with  a  code  complex. 
While  10  different  code  sets  were  used  in  our  experiments,  there  were  only  six 
basic  form  sets.  One  of  these  sheets  was  randomly  selected  for  each  of  the  six 
form  codes. 

Experimental  packets  consist  of  a  cell  matrix  response  form  and  copies  of  the 
six  code  displays.  Target  categories  were  listed  across  the  top,  grouped  as  in  the 
heading  of  Table  11.  Code  sets  were  identified  along  the  left  side  of  the  matrix. 


Subjects 

Two  of  the  Human  Engineering  classes  at  these  laboratories  participated  in 
this  experiment,  providing  37  subjects.  Fifteen  of  our  or  hnal  subjects  were  also 
used.  Following  a  lecture  on  the  problems  of  coding  information  for  an  air -defense 
display,  these  groups  were  asked  to  help  us  identify  the  best  choice  of  meaning  for 
symbols  of  six  proposed  codes. 

The  experimenter  gave  each  subject  an  experimental  packet  and  explained  the 
basic  design  of  each  code  set.  He  emphasized  that  the  sheets  demonstrated  only 
the  general  appearance  of  the  coding  alphabets  and  that  target  position  should  not  be 
considered  as  a  threat  evaluation.  He  then  asked  the  subjects  to  study  the  displays 
and  sketch  their  recommended  coding  symbols  in  the  cells  of  the  matrix. 

This  procedure  was  completed  rapidly  with  only  a  few  questions.  Subjects 
were  then  asked  to  indicate  the  code  that  seemed  most  suitable. 

The  experiment  sparked  considerable  interest  and  a  rather  spirited  discussion. 
There  was  general  agreement  on  the  fact  that  friendly  and  hostile  categories  each 
called  for  symbols  that  paired  easily.  This  meant  that  the  unknown  meaning  was 
assigned  to  the  "odd-ball"  symbol  within  each  set. 

Results  are  tabulated  in  Table  11. 
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Results 


With  52  subjects  and  five  meanings,  a  chance  distribution  of  choices  would 
call  for  52/5  or  10.4  assigned  to  each  meaning  category.  This  would  reduce  to  10.4 
within  a  single  category  versus  41. o  outside  that  category.  This  one  degree  of 
freedom  design  for  a  chi  square  test  of  significance  reduces  to: 


where  d^  =  assignments  within  category  -  10.4 

d2  =  assignments  in  other  categories  -  41.6 
and  dj  =  d2 

2  2 

With  one  degree  of  freedom  (one -tail  test)  X  -  5.0  is  significant  at  P  =  .05;  X  =8.1 
is  significant  at  P  =  .01.  Substituting  these  values  in  the  above  equation  we  find 
that  the  .05  level  is  reached  with  16.8  assignments  and  the  .01  level  requires  18.6 
choices.  On  Table  11  selection  at  these  levels  have  been  indicated  as  *  and  **  follow¬ 
ing  normal  conventions. 

Selections  for  the  two  friendly  categories  are  clear-cut  and  follow  our  suggested 
meanings.  There  is  a  similar  trend  in  selection  for  the  unknown  target  in  all  excel* 
set  UIc. 

Hostile  targets  as  a  class  arc  less  meaningful  to  our  subjects.  The  diamond 
and  pointer  have  a  strongly  hostile  meaning  in  all  uses.  Choice  of  the  pointer  as 
a  hostile  missile  is  significant.  The  diamond  is  more  strongly  hostile  than  either 
the  square  or  plus  sign  of  the  HEL  designs.  The  lexicon  is  not  strongly  meaningful 
outside  the  friendly  meanings . 

In  choosing  the  best  symbol  code,  three  subjects  had  no  preference.  Choices 
were  divided  as  follows:  nine  subjects  preferred  Series  I,  three  chose  Series  II 
symbols  with  the  half -circle,  four  chose  Series  II  with  the  ellipse.  The  two  HEL 
designs  drew  27  votes,  with  15  for  the  set  with  a  square  and  12  for  the  plus -sign 
grouping.  Six  choices  went  to  the  logical  lexicon. 

Tests  of  significance  on  this  population  would  be  doubtful  validity,  since  there 
is  no  logical  reason  for  grouping  the  small  populations.  Preference  trends  lean  in 
the  direction  of  the  HEL  designs. 
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Discussion 


This  experiment  leads  further  weight  to  our  earlier  experiments  on  symbol 
meaning.  It  also  noints  up  the  fact  that  we  have  been  quite  successful  in  selecting 
friendly  forms,  -iostile  symbols  need  further  study  for  ideal  selection. 


CONCLUSIONS  AND  RECOMMENDATIONS 
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Coding  methods  for  radar  displays  are  aimed  at  enhancing  information.  Any 
code  which  does  not  provide  an  advantage  in  time  and  accuracy  is  an  inadequate 
substitute  for  raw  radar  returns.  Our  studies  have  been  aimed  at  defining  the 
factors  necessary  for  designing  a  symbolic  code  which  improves  the  rapid  and 
accurate  assimilation  of  available  information. 

The  current  studies  at  HEL  have  been  directed  at  recognition  of  symbols  in  a 
mixed  display,  representative  of  an  air-defense  picture.  Our  findings  lead  to  fairly 
strong  conclusions: 

1.  Discernibility,  or  pure  visibility,  of  forms,  is  not  an  adequate  measure 
for  selecting  symbols  to  code  mixed  information.  Two  or  more  highly  discernible 
forms  may  interact  with  resultant  poor  discriminahility. 

2.  Any  experimental  method  which  presents  symbols  individually  cannot 
demonstrate  interaction  between  symbols.  Sorting  techniques  are  not  suitable  for 
code  selection. 

3.  Experiments  using  black -on -white  drawings  representative  of  a  coded, 
five -dimensional  PPI  display  suggest  the  following  tentative  rules  for  code  design: 

a.  Symbols  should  differ  strongly  in  shape.  Variations  of  a  single  form 
family  such  as  the  circle  and  ellipse  are  not  desirable. 

b.  Redundant  cues  such  as  size  difference  may  be  advantageous. 

c.  Characteristics  which  are  enhanced  by  a  unique  search  method  add 
to  the  saliency  of  an  individual  symbol  in  a  code  complex. 

d.  If  several  basic  symbols  are  to  be  grouped  within  a  major  classifica¬ 
tion,  a  familial  resemblance  maybe  desirable.  Absolute  discrimination  of  these 
basic  forms  requires  that  the  resemblance  be  descriptive  rather  than  perceptive. 

If  absolute  discrimination  is  not  required,  a  single  symbol  should  be  adequate. 

f.  The  most  satisfactory  five -dimension  code  in  the  current  studies 
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(Series  Ilia)  results  in  an  average  locating  time  of  0.80  seconds  combined  with  an 
average  error  rate  of  0.44  percent.  The  poorest  code  (Series  IIIc)  requires  an  average 
average  locating  time  of  1.34  seconds  with  2.64  percent  errors.  This  advantage 
reduces  time  to  59  percent  and  errors  to  16  percent. 

4.  Further  experiments  using  actual  CRT  displays  are  recommended  before 
final  standardization  of  radar  symbols.  A  variety  of  phosphors  and  ambient 
il'umination  levels  should  be  included. 

5.  Learning  studies  should  be  conducted  to  select  ideal  methods  for  increasing 
the  informationai  value  of  the  basic  codes.  Modifiers  added  to  the  basic  symbols 
could  confuse  the  total  picture.  A  trade-off  between  the  advantages  of  total  informa¬ 
tion  in  a  single  symbolic  unit  and  clean  symbols  with  auxiliary  read-outs  for  additional 
information  must  be  considered. 
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APPENDIX  A 


I 


Questionnaire  Following  Series  I 


You  have  just  completed  a  test  which  represents  some  of  the  problems  a  radar 
operator  may  have  if  he  is  given  display  information  in  symbolic  coding.  Each  shape 
represents  an  individual  target  class  and  the  various  modifiers  could  offer  additional 
information  such  as  speed  or  priority. 

Since  the  operator  will  be  working  rapidly  with  a  large  amount  of  information, 
the  codes  must  be  kept  simple  to  remain  useful.  We  would  appreciate  your  help  in 
defining  problems.  Please  answer  the  following  questions,  explaining  your  reasoning. 
We  would  also  appreciate  any  comments  that  come  to  your  mind. 

1.  Did  you  find  this  test  difficult?  _  Why? _ 


2.  Which  of  the  four  sets  seemed  easiest? _ __ 

Which  did  you  find  most  difficult? _ _ 

3.  Did  you  notice  any  patterning  of  targets?  _  Did  you  try  to  find 

a  pattern  as  you  searched?  _ 

4.  Which  shapes  were  easiest  to  locate?  _ 

5.  Were  any  shapes  especially  difficult? 

6.  Can  you  suggest  changes  that  would  make  a  task  of  this  type  easier?  _ 


7.  Do  you  think  of  anything  that  might  make  location  of  targets  excessively 
difficult? 


PLEASE  ADD  FURTHER  SUGGESTIONS  OR  COMMENTS. 


b 
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TABLE  IB 

Statistical  Pattern  for  Time  Analysis 
Series  lb,  N=20,  k=5 
Analyses  of  Variance 


Source  of 
Variation 

Sum  of  Squares 

Degrees  of  Freedom 

Mean  Square  F 

Between  People 

4.2618 

19 

Within  People 

4.4233 

80 

Shape 

3.4485 

4 

.86213 

67.20** 

Residual 

0.9748 

76 

.01283 

Total 

8.6851 

99 

**F>99  (4,76) 

=  2.19 

Newman-Keuls  Test  of  Differences 

Half 

Shape 

Circle 

Circle 

Square 

Triangle 

Diamond 

Totals  16.67 

19.06 

24.63 

24.87 

26.14 

Half  Circle 

16.67 

2.39** 

7.96** 

8.20** 

9.47** 

Circle 

19.06 

- 

5.57** 

5.81** 

7.03** 

Square 

24.63 

- 

0.24 

1.51 

Triangle 

24.87 

- 

1.27 

Diamond 

26.14 

*  q  (r,  76)  N  M  S  resid. 

1.43 

1.72 

1.89 

2.01 

**q  .99 (r,  76)  N  M  S  resid. 

i.90 

2, 16 

2.32 

2.43 

max 


■  0697 
.0472 


i.48  lFmax  .95(5,  19)  =  3.71] 
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’This  report  covers  continued  studies  toward  standardization  coding  symbols  for  an 
information  display.  Experiments  were  directed  toward  locating  "the  most  readily 
discriminated  five -symbol  code  com^x"  as  measured  by  errors  and  location  times. 

Experiment  if  attempted  to  simplify  testing  procedures  by  using  a  card  sorting  task. 
The  same  five -symbol  code  was  presented  as  a  black -on -white  simulated  display  in 
Experiment  II;  Results  were  not  comparable  and  the  simulated  display  was  used  in 
further  experiments  with  a  variety  of  codes. 

Legibility  and  association  values  of  individual  forms  varied  with  the  puliation  of 
shapes  within  the  code  complex.  Experimental  results  led  to  general  recorr.me  Ha  .io  ; 
for  code  design.  Further  studies  are  anticipated. 


